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Preface 



The fourth Algorithmic Number Theory Symposium takes place at the Uni- 
versiteit Leiden, in the Netherlands, from July 2-7, 2000. Its organization is a 
joint effort of Dutch number theorists from Leiden, Groningen, Nijmegen, and 
Amsterdam. 

Six invited talks and 36 contributed talks are scheduled. This volume contains 
the written versions of the talks, with the exception of two of the invited talks. 
Not included are: A rational approach to tt by Frits Beukers (Utrecht) and The 
40 trillionth binary digit of tt is 0 by Peter Borwein (Burnaby, Canada). These 
talks are aimed at a wider audience, and form part of the special ANTS IV event 
Pi in de Pieterskerk on July 5, 2000. This event includes an evening ceremony 
in which the tombstone of Ludolph van Ceulen is replaced. Van Ceulen, who 
was appointed to Leiden in 1600, calculated 35 decimals of tt. His tombstone in 
the Pieterskerk, in which these decimals were engraved, disappeared in the 19th 
century. 

ANTS in Leiden is the fourth in a series of symposia that started in 1994. 
Previous locations were Cornell University, Ithaca, New York (1994), Universite 
de Bordeaux I in Bordeaux, France (1996), and Reed College, Portland, Oregon 
(1998). The diversity of the papers contained in this volume shows that the 
main theme of ANTS, algorithmic number theory, is taken in a broad sense. The 
number of submissions for the Leiden conference largely exceeded the physical 
limitations of our one-week schedule. We are therefore confident that we are only 
at the beginning of a continuing tradition. 



May 2000 Peter Stevenhagen 

ANTS IV Program Chair 
Wieb Bosma 
Proceedings Editor 
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The Complexity of Some Lattice Problems 



Jin-Yi Cai* 

Department of Computer Science and Engineering 
State University of New York, Buffalo, NY 14260, USA 
caiScse .buffalo .edu 



Abstract. We survey some recent developments in the study of the 
complexity of certain lattice problems. We focus on the recent progress 
on complexity results of intractability. We will discuss Ajtai’s worst- 
case/average-case connections for the shortest vector problem, similar re- 
sults for the closest vector problem and short basis problem, NP-hardness 
and non-NP-hardness, transference theorems between primal and dual 
lattices, and application to secure cryptography. 



1 Introduction 

Mostly stimulated by the recent work of Miklos Ajtai, there has been renewed 
interest and activity in the study of lattice problems. Research in the algorith- 
mic aspects of lattice problems has been active in the past, especially following 
Lovasz’s basis reduction algorithm in 1982. The recent wave of activity and in- 
terest can be traced in large part to two seminal papers written by Miklos Ajtai 
in 1996 and in 1997 respectively. 

In his 1996 paper [1], Ajtai found a remarkable worst-case to average-case 
reduction for some versions of the shortest lattice vector problem (SVP), thereby 
establishing a worst-case to average-case connection for these lattice problems. 
Such a connection is not known to hold for any other problem in NP believed to 
be outside P. In his 1997 paper [2], building on previous work by Adleman, Ajtai 
further proved the NP-hardness of SVP, under randomized reduction. The NP- 
hardness of SVP has been a long standing open problem. Stimulated by these 
breakthroughs, many researchers have obtained new and interesting results for 
these and other lattice problems [3,10,14,15,16,17,18,19,20,26,32,33,34,35,36,55], 
,[58,61]. Our purpose in this article is to survey some of this development. 

In my view these lattice problems are intrinsically interesting. Moreover, 
the worst-case to average-case connection discovered by Ajtai also opens up 
possibilities regarding provably secure public-key cryptography based on only 
worst-case intractability assumptions. It is well known that the existence of 
secure public-key cryptosystems presupposes P yf NP. However the converse 
is far from being proven true.^ The intractability required by cryptography is 

* Research supported in part by grants from NSF CCR-9634665 and a John Simon 
Guggenheim Fellowship. 

^ I do not want to say “the converse is false”, since it is probably true for the reason 
that both P yfNP and there exist secure public-key cryptosystems. But it is believed 
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more concerned with average-case complexity rather than worst-case complexity. 
Even if we assume that some problem in NP is not solvable in P or BPP, this 
still leaves open the possibility that the problem might be rather easy on the 
average. 

Consider the security of RSA and the intractability of factoring. First, we 
do not know if factoring is not solvable in P or BPP. We do not know if this 
is so assuming P yf NP. We do not even know whether it is NP-hard. Second, 
even if we assume it is NP-hard or not solvable in P or BPP, we do not know 
it is as hard for the special case of factoring a product of two large primes p ■ q. 
Third, even if factoring p ■ q is hard in the worst case, we do not know if it 
is hard on the average, under some reasonable distribution on such numbers. 
Fourth, we do not know if decrypting RSA without the private key is equivalent 
to finding (p{pq) = {p — l){q — 1), (although given n = p ■ q, finding (p{pq) is 
equivalent to factoring). Thus although RSA is believed to be an excellent public- 
key cryptosystem, there is a large gap between the assumption that factoring 
is hard in the worst-case (say it is not in BPP) and a proof that the system is 
secure. 

Building on Ajtai’s worst-case to average-case connection, Ajtai and Dwork 
[3] proposed a public-key cryptosystem that is provably secure, assuming only the 
worst case intractability of a certain version of SVP, namely to find the shortest 
lattice vector in a lattice with n'^-unique shortest vector, for a sufficiently large 
c. This is the first time that such a provable security guarantee based on the 
worst-case complexity alone has been established. However, for the important 
topic of application to Cryptology, Nguyen and Stern have written an excellent 
survey appearing in these proceedings [59] . Therefore I will not discuss this topic 
in any detail here and refer to [59]. 

In Section 2 we collect some definitions. I will then discuss Ajtai’s worst- 
case/ average-case connection for the shortest vector problem, and the worst- 
case/ average-case connection for related closest vector problem (Section 3), NP- 
hardness results (Section 4), evidence of non-NP-hardness via bounded round 
interactive proof systems (Section 5), and transference theorems relating primal 
and dual lattices (Section 6). 

I am sure many important works have been neglected or not given its proper 
due. I apologize for any such omissions or mistakes. 

2 Preliminaries 

A lattice is a discrete additive subgroup in some M”. Discreteness means that 
every lattice point is an isolated point in the topology of M". An alternative 
definition is that a lattice consists of all the integral linear combinations of a set 
of linearly independent vectors, 

L = riibi I rii € Z, for all z}, 

i 

that it is insufficient to assume only P t^NP in order to prove pseudorandom number 
generators exist. 
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where the vectors bi’s are linearly independent over M. Such a set of generating 
vectors are called a basis. The dimension of the linear span, or equivalently 
the number of bi's in a basis is the rank (or dimension) of the lattice, and is 
denoted by dimL. We may without loss of generality assume that dimL = n, for 
otherwise we can replace M” by its linear span. We denote L as L(6i, 62, . . . , bn)- 

The basis of a lattice is not unique. Any two bases are related to each other 
by an integral matrix of determinant ± 1 . Such a matrix is called a unimodular 
matrix. Clearly an integral matrix has an integral inverse iff it is unimodular, 
following Cramer’s rule. 

The parallelepiped 

P{bi, ...,&„) = Xibi I 0 < a;i < 1 } 

is called the fundamental domain of the lattice. 

Since basis transformation is unimodular, the determinant | det(6i, . . . , &„)| 
which is the volume of the fundamental domain P{bi , . . . , b„) is independent of 
the basis, and is denoted by det(L). 

We use Isp to denote linear span over M. Given a basis {61, 62, . . . , bn} of L, 
let Pi = lsp{ 5 i, ... ,bi} he the linear span of {bi, . . . , bi}, and Li = L{b\, . . . ,bi) 
be the sublattice generated by { 5 i, . . . , bi}. We denote by 11:^ the orthogonal 
complement of II i. The process of Gram-Schmidt orthogonalization obtains from 
a basis {61, 62, . . . , bn} a set of orthogonal vectors {61, 62, . . . , 6„}, where bi is 
the orthogonal component of bi perpendicular to Ili-i'. 



bi = h~Y. 

j<i 



{bj, bj) '^ 

(bj,bj) 



1 < z < n, 



where (•, •) denotes inner product. 

The fundamental domain as well as the orthogonal “brick” P{bi , . . . , 6„) = 
[ 0 , 61) X • • • X [ 0 , bn) form a tessellation of M” by translation. We can also tessellate 
M” by the centralized “brick” B = [~\,\) x • • • x 

M" = \J{e+B). 

ieL 



We note that the volume vol D = vol B = det L. 

The length of the shortest non-zero vector of L is denoted by \\{L). In 
general, Minkowski’s successive minima Xi{L) are defined as follows: for 1 < z < 
dimL, 



Xi{L)= min max II zzj 1 1 , 

. ,Vi^L 

where the sequence of vectors vi, . . . ,Vi € L ranges over all z linearly independent 
lattice vectors. It is not difficult to show that to get Vi G L with ||z;j|| = Aj, 
one can always take greedily any linearly independent vi,. . . ,Vi-i G L, with 
Ikill = Ai, . . . , ||ui-i|| = Ai_i. 
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Let L be an n-dimensional lattice in R” with basis {bi, 62 , . . . , bn}- Since the 
translations of the fundamental domain D = P{bi, 62 , , b„) form a tiling of 
M”, the volume vol(I?) = det(L) provides a certain measure of the size of L. 
Minkowski’s First Theorem makes an explicit connection of the shortest lattice 
vector and this quantity [57,24,39]: 

Theorem 1 (Minkowski). 

Ai(L) < 7 „(det(L))i/", 

where 7 „ is some universal constant. 

The smallest such constant for dimension n is denoted by 7 „ and called Her- 
mite’s constant of rank n. Minkowski proved that 7 „ < -^r{^ + 1)^/", which 

is asymptotically It is known that < 7 n < The upshot is, for 

a lattice with det(L) = 1 , (after a suitable scaling), there is always a non-zero 
short vector of length no more than ^/n. 

Minkowski’s First Theorem has a short and elegant proof: Consider the lattice 
L' = 2L, which is a dilatation of L by a factor of 2 in all directions. det(L') = 
2” det(L). Consider a ball of radius r centered at every lattice point of L' . Let 
denote the volume of a unit ball Bn, then is the volume of a ball i?„(r) of 
radius r. Now if > det(L^), there must be some overlap among two different 
balls, thus yf i' both G L, such that + x = 2S!! + y for some x, y G Bn{r). 
Then £ — £' = {y — x)/2 G Bn{r) by convexity. And £ — £' is our non-zero lattice 
point of L. It is known that = 7 t”/^/T(.| -|- 1). It follows that 

Ai(L) < ^T(5 + l)i/"(det(L))i/" = 0(VH)(det(L))i/". 

Y 7T Z 

Theorem 1 follows. 

A more general theorem, also due to Minkowski, is concerned with successive 
minima: 

Theorem 2 (Minkowski). 

/ n \ 1 /" 

fnA(L)j < 0 (V^)(det(L))i/". 

While Minkowski’s theorem guarantees the existence of vectors as short 
as v^det(L)^/”, there is no polynomial-time algorithm to find such a vector. 
Minkowski’s proof is decidedly non-constructive. The Shortest Vector Problem 
(SVP) is the following: Given a basis of L, find a vector v G L such that 
Ijujl = Ai(L). One can also define various approximate short vector problems, 
seeking a non-zero v G L with ||u|| bounded by some approximation factor, 
\\v\\ < /(n)Ai(L) or ||u|| < /(n)(det(L))i/". 

We denote by bl(L) the basis length of L 

bl(L) = min max||5i||. 

all bases for L *=1 
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The dual lattice L* of a lattice L of dimension n in R" is defined as those vec- 
tors u e R", such that (u, v) G Z, for all v G L. For a basis {6i, 62 , . . . , bn} of L, 
its dual basis is {6*, 63 , . . . , bn}, where {b*,bj) = Sij. Then L* = L(5*, 63 , . . . , bn). 
In particular det(L*) = l/det(L), and L** = L. For a lattice with dimension 
less than n, its dual is defined within its own linear span. 

We let kL = {kv \ V G L} be the dilatation of L for any positive fc G R. Let 
X + A = {x + y \ y G A} ior any a; G R" and A C R". Let A + B = {a + b\ a G 
A,b G B}. We denote by [xj the greatest integer < x, [a;] the least integer > x, 
[a;] = — a;J, and [a;] the closest integer to x, [a;] = [a;-l- 

3 Ajtai’s Worst-Case to Average-Case Connection 

Let n, m and q be arbitrary integers. Let denote the set of n x m matrices 

over Zq, and let fin,m,q denote the uniform distribution on Z”^™. For any X G 
Z"xm^ the set A(X) = {y G Z™ | Xy = 0 mod q} (where the congruence is 
component-wise) defines a lattice of dimension m. Let A = An^m,q denote the 
probability space of lattices consisting of A{X) by choosing X according to 

^n.m.q- 

We note that indeed A{X) is a lattice of dimension m, since it is clearly a 
discrete additive subgroup of Z™, and each qa G A(X), where Cj has a single 
1 at the zth position and 0 elsewhere. It also follows that A(X) repeats itself 
within each q x q x ■ ■ ■ x q box. In other words, A{X) is invariant under the 
translations y y + qei, for each 1 < z < m. 

By Minkowski’s First Theorem, it can be shown that 

Vc 3c' s.t. Vyl(3f) G An,c’n,n<= 3zz (v G A{X) and 0 < ||z;|| < zz). 

In fact the bound zz can be reduced to zz^+*^. The bound ||z;|| < zz is needed 
to ensure that the assumption on the hypothetical algorithm A below is non- 
vacuous. 

Theorem 3 (Ajtai). Suppose there is a probabilistic polynomial time algorithm 
A such that for all zz, when given a random lattice A{X) G An,m,q where zzz = 
an log zz and q = for appropriate constants a, (3, returns with probability ^o\i) ? 
a vector of A{X) of length < zz, then there exists a probabilistic polynomial time 
algorithm B such that for all zz, when given a basis {oi, . . . , a„} for an arbitrary 
lattice L = L{a \, . . . , a„), performs the following with high probability: 

1 ) Finds a basis {bi , . . . , &„} for L such that 

max|j6i|| < zz'’^ •bl(L), 

i—1 

2) Finds an estimate A of Xi{L) such that, 
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3) Finds the unique shortest vector ±v of L, if L has an unique shortest 

vector, i.e. \ 2 {L) > • \\{L), 

where ci,C 2 ,C 3 are absolute constants. 

Remark: This is the first such worst-case to average-case connection proved for 
a problem in NP believed not in P. While random-self-reducibilities were known 
for other problems, such as Quadratic Residuosity (QR), there is a technical 
difference. In QR, one must fix a modulus, then there is a worst-case to average- 
case connection for this modulus. But no such reduction is known among different 
moduli. The permanent is another example where there is a certain worst-case 
to average-case connection (see [31,30,21,38]), but the permanent is not believed 
to be in NP. 

Items 2) and 3) are derived from item 1) via a transference type argument, 
about which we will say more later in Section 6. Here we will focus on the ideas 
in the proof of item 1). Without loss of generality, we can assume that the lattice 
consists of integral vectors. The same result also holds for lattices with rational 
entries or with entries from any subfield of C, as long as there is an effective bit 
representation for the lattice. 

We will now present some ideas from the proof. 

Suppose we currently have a basis {bi, . . . , &„}, where max(bj^ ||6i|| is greater 
than bl(T) by a large polynomial factor i.e. 

qi =^g£iMx||6i|| > n^ibl(L). 



The main procedure of B is iterative. Let S' be a set of n independent vectors 
of L (initially S = {bi, . . . , &„}). If the length of the elements of S at the start 
of the current iteration is large enough, the algorithm finds a set of independent 
vectors, each of at most half the length, with high probability. This means, in 
a polynomial number of steps we will have a set of short enough vectors, which 
can then be converted to a short basis with a loss of a factor < ^/n. 

The fundamental domain D = P{b\, . . . , &„) forms a tiling of M" via trans- 
lations under L, 

M" = + 

l£L 



as a disjoint union. 

Consider a large cube 



n 

Q = {x e M" I X = ^ Xi€i,0 <Xi< M}, 

i=l 

where M is a certain polynomial factor greater than qi, say, M = n~* qi. For 
each i, we can “round” the corner point Mci to a lattice point according to 
which translate -I- I? it belongs to. This only involves solving a linear system 
expressing Mci as a rational linear combination of the basis {bi , . . . , b„} and 
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then rounding the coordinates. Thus for each i = 1, . . . ,n, let 



n n 

Mg = ajjbj and k = 

i=i i=i 



Now 



n 

= {a; C R” I a; = ^ Xik, 0 < Xi < 1}, 

i=l 



is a reasonably good approximation of Q; we will call it a pseudocube. Note that 
the corner vertices of Q' are all lattice points. To ensure that Q' looks reasonably 
close to a cube, Ajtai chose 7 = 3. 

In the next step we subdivide Q' into a family of disjoint sub-pseudocubes, by 
subdividing Q' along each direction li into q subintervals, where q is polynomially 
bounded in n. 



0<ki,... ,kn<q 



where the basic sub-pseudocube 



Q" = {a; G 



X = 'Y' Xik, 0 <Xi < 






We will make sure that the length of a side of Q" , which is roughly -y, is still 
larger than bl(T) by a significant polynomial factor. 

Suppose this is the case. Then with a series of technical lemmas, Ajtai shows 
that the number of lattice points within each translate Q" + is roughly 

the same. This is intuitively quite plausible. But the technical details are not 
straightforward, especially if one wants a reasonably good bound. (See below.) 

Once this approximate equi-distribution of lattice points is established, one 
can sample the “addresses” (fci, . . . , of sub-pseudocubes, by uniformly sam- 
pling a lattice point in Q' . Once a lattice point v is picked, we decide to which 
sub-pseudocube it belongs by expressing u as a linear combination Y^=i 
where Q < Xi < q, hy solving a linear system. Then, we round off Xi and set 
ki = [a;ij . 

More generally, suppose we get m such samples, Vj & L, 1 < j < m. We 
decompose Vj as follows, (See Figure 1) 









where rj is a vector in Q" . Note that ||rj ||2 is 0{ 
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Figure 1 



Here is the key observation: Suppose we are able to obtain an integral solution 
■ ■ ■ ) Cm) to 

m 

kij^j = 0 mod q, 
i=i 

then would be a lattice point which has an interesting decomposition, 








( 1 ) 



We note that the quantity — ^ is actually an integer, which makes the 
first term in (1) a lattice vector. Hence being the difference of two 

lattice points, must be a lattice point itself, (even though each rj is probably 
not a lattice point.) 

Suppose the integral solution X has every |^j| < n, then 



I I All \ 

i=i ^ 



= O 



, 1 . 5+7 



( 2 ) 



Now q can be chosen 6 >(n®) so that |j J2jLi < f > which is at most half of 
every ||5i||. 

With the choice of 7 = 3, Ajtai showed that the shape of the pseudocube 
and thus that of the sub-pseudocubes is very close to a perfect cube. With a 
choice of q = and a corresponding m = O(nlogn), Minkowski’s Theorem 

applies. Hence the assumption on A is non- vacuous and the newly produced 
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lattice vector J2jLi length < On the other hand, the length of a side of 

a sub-pseudocube is approximately ^ which is bounded below by ^ bl(L) = 
6>Ki-3bl(L)). 

With the shape of the pseudocube approximately a perfect cube, and with 
a sufficiently large ci , which makes each side of the sub-pseudocube sufficiently 
larger than bl(L), Ajtai showed that the distribution induced on the address 
space {(fci,... ,kn) | 0 < fci < g} by uniformly sampling lattice points from 
L is close to uniform. In fact, not only must the distribution of each sample 
(fci, . . . , kn) be close to uniform, but also the joint distribution on all the m sam- 
ples forming the matrix (kij) must be close to the uniform distribution f2n,m,q- 
Only then can one legitimately invoke the assumed algorithm A and be guaran- 
teed to obtain a short vector X with = 0 niod q, and IjAH < n, with 

nontrivial probability. 

So far we have only produced one lattice vector b'l = which is 

shorter than ^ = max |j6i|| by a factor of 2. We continue this process to produce 
n linearly independent lattice vectors {6 ^, . . . , b'„} to replace {6 i, . . . , &„}. To 
show that these successive b^ are linearly independent demands another set of 
technical lemmas which ultimately depend on the fact that ci is sufficiently large. 
In that case, Ajtai showed that within each sub-pseudocube the lattice is quite 
dense. It follows that, for every n — 1 dimensional hyperplane 7T, the number of 
lattice points on 7T n Q" is much smaller compared to the total number of lattice 
points in Q” . Moreover, this is true for every translate of Q” . It follows that the 
successive 6'’s are not likely to be linearly dependent on {6 ^, . . . ,b'i-i\- We will 
not provide any more technical details of Ajtai’s proof. The interested reader is 
referred to [1]. 



Improving Ajtai’s Connection Factors 

What is outlined above is essentially Ajtai’s proof [1], where some universal 
constants ci, C2 and C3 are shown. Although no explicit values for these Cj’s were 
given, and apparently no special effort was made to minimize them, implicitly a 
factor less than 8, 10 and 19, respectively, can be derived from the proofs of [1]. 

The factors are called Ajtai’s connection factors; they provide a mea- 
sure of the tightness of the worst-case to average-case connection. The smaller 
the constants are, the tighter the connection one gets. As 2) and 3) are derived 
through 1) (see Section 6), is the crucial factor. Cai and Nerurkar [19] ob- 
tained a substantial improvement to , and consequently to the other factors 
as well. Here we give an overview of some of the ideas involved in this improve- 
ment. As is the case with Ajtai’s proof [1], there are a number of technical points 
we have to gloss over due to limited space. 

The general structure of the procedure of Cai and Nerurkar [19] closely follows 
Ajtai’s proof, but much of the technical justification is different. As we saw above, 
the general idea is to sample lattice points, in order to induce an almost uniform 
distribution on a set of “address” vectors, which form the columns of a matrix 
that is close to uniformly distributed. The assumed algorithm A is applied to 
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this matrix. By hypothesis, this algorithm performs well on the average, and 
thus we get a short vector which can be turned into a short vector of the original 
lattice. 

In the choice of M = we need 7 to be a sufficiently large constant in 
order to ensure that the resulting pseudocube is reasonably close to a perfect 
cube. We call this the shape condition. Then, we need to choose an integer q 
to be a sufficiently large polynomial (in n) in order to ensure that the newly 
produced remainder vector is shorter than the previous 1 1 6^ 1 1 . This involves m in 
the numerator in q in ( 2 ), which has to be chosen after q in order to 

ensure that short vectors exist by Minkowski’s First Theorem. Fortunately, this 
is not circular; for any polynomially bounded q, m only needs to be 0{n). But 
still q must depend on 7 . Finally, given q, we must ensure that the length of a 
side of a sub-pseudocube M/q is sufficiently large compared to bl(L). We know 



that. 



M _ 
q q 



q 



hl{L) 



This is where /i > rf^h\{L) is used and ci has to be large. Cai and Nerurkar [19] 
achieve c\ = 3 -I- e for linearly independent vectors, and c\ = 3.5 -I- e for basis 
length. 

The algorithmic improvement by Cai and Nerurkar [19] starts with a tiling 
of M” by orthogonal “bricks” of sides at most /i, via Gram-Schmidt orthogo- 
nalization. This is in contrast to the tiling by fundamental domains in [1]. The 
advantage is that one can round off from a perfect cube to a lattice pseudocube 
with less error. Thus, for M = and Wi = Met, we can round off Wi 

to a lattice point k such that Wi = k + Si and |](5i|l < This implies 

ll^ill < P(h , . . . , In) is the pseudocube constructed. 



h 




Secondly, in [19], the pseudocube is positioned centrally and subdivided. Each 
sub-pseudocube will have an address vector at the center. More precisely we will 
take Q' = P(2li, , 2l„) - J2i=i = {S"=i I - 1 < < 1 }. We partition 

Q' into g” sub-pseudo-cubes, (where q is odd, say), such that the basic sub- 
pseudocube is Q' = {X^r=i I ~ q — ^ Sample lattice points 

uniformly in the pseudocube Qb This induces an almost uniform distribution 
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on the address space. But this time we consider each address as corresponding 
to the center of the sub-pseudocube. When we express a sample lattice point vj 
as the sum of this address vector and a remainder vector rj, these remainder 
vectors tend to be symmetrically distributed with respect to the address vector 
at the center. (See Figure 2) 

Here an address vector is of the form X)r=i where each kij is even, 

— (9— 1) < kij < q—l. The corresponding “address” is (fcij, k 2 j, ■ ■ ■ , knj) reduced 
modulo q. Thus, when we estimate || ^j'^jW probabilistically, the indepen- 

dent Tj’s tend to cancel out instead of adding up. Note that X = (^1 , . . . , 
is a (short) solution obtained by the algorithm A given only the address matrix 
(kij). Given such a matrix one must ensure that the Vj are almost independently 
and centrally symmetrically distributed. This is geometrically quite intuitive, 
given a sufficiently large ratio of the sides of the sub-pseudocube to bl(L). But 
the hard part is to minimize this notion of “sufficiently large” . It turns out that 
q = and n > n^“*'*^bl(L) will do. The technical part of the proof is rather 
involved. 

There is one more idea in [19] in the improvements in terms of the algorithmic 
steps. It turns out to be insufficient to guarantee the generation of one almost 
uniform address vector, which makes up one column of the matrix. We must 
be able to generate m columns to form an almost uniformly generated matrix. 
This more stringent requirement is needed to apply the algorithm A. In [19] we 
used an idea to amplify the “randomness” in each column vector generated, by 
adding together [2/e] copies of independent samples 



ki . 



= '^—h + r, 



2=1 ^ 
k' 

v' = y^ ^k + r', etc. 

^ q 



i=l 



This gives a lattice point 

n 

v+v '^ — = 



ki -\- k^i ' 



-k + (r + r' -\ ). 



i=l 



Starting from the column vector (fci, ^2, ■ ■ ■ 7 kn) being n “^-close to uniform, we 
show that the address vector 



{ki + k[ + ■■■, k 2 + k '2 + kn + k'n + ■■ ■) mod q 



is n“^-close to uniform, which would be sufficient to ensure that the matrix is 
close to being uniform. The price we pay for this is that each remainder vector 
is enlarged by a factor at most [2/e]. 

The more difficult part of the proof is to show that the lattice samples do 
induce a distribution that is n“'^-close to uniform on the address space. In ad- 
dition to our “shape condition”, which is accomplished by 7 = 1.5, we need to 
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estimate the volume of each sub-pseudocube to ensure that the number of lat- 
tice points within each sub-pseudocube is almost identical. Moreover, in order 
to obtain independent lattice vectors, we need to ensure that the proportion of 
lattice points in a sub-pseudocube that lie on any (co-1 dimensional) hyperplane 
is negligible. 

The bounds in [19] use eigenvalues and singular values, and a theorem of 
K. Ball [7]. We cannot go into much detail here, but the following lemmas give 
a flavor of it. 

Lemma 1. Let ei, . . . , e„ he the standard unit vectors. Let ui, . . . , he lin- 
early independent vectors such that \\ui — eijj < e. Then the parallelepiped 
V{ui , . . . , Un) has volume 

I -ne < vol{V{ui, . . . , m„)) < (1 -f e)”. 

(One cannot improve the lower bound to (1 — e)” for large n.) 

Lemma 2. Let ei, . . . , e„ and u\, . . . ,Un he as above. Let H he a hyperplane. 
Then the {n — 1)- dimensional volume of P{u \, . . . , rt„) n is at most -\/2e(l + 
e)"-b 



A Worst- Case/ Average- Case Connection for CVP 

The best bound for the hardness of CVP is by Dinur et. al. [26]. They show 
that CVP is NP-hard to approximate within a factor 2*°®^ ' ”, for an e = o(l). 
Goldreich et. al. [36] show a direct reduction from SVP to CVP. This reduction 
has the property of preserving the factor of approximation for the two problems 
and the dimension of the lattice. 

A corresponding result of worst-case/ average-case connection for CVP was 
established by Cai [23] recently. Note that the known NP-hardness reductions 
to CVP do not provide any evidence of hardness for the average-case complexity 
of CVP. This is generally true for NP-hardness reductions, since the reductions 
only produce very specialized instances of the target problem, in this case CVP. 



Theorem 4. Lf there is a prohahilistic P-time algorithm A, for a uniformly 
chosen lattice L in the class A indexed hy n and a uniformly chosen target vector 
u, A finds a lattice vector v G L such that ||u — u|| < n, with prohahility at least 
then, there is a prohahilistic P-time algorithm B, for any lattice L' of 
dimension N, with prohahility 1 — e~^ , will 

— For any target vector x find a lattice vector y G L' with distance [[x — y|| < 

A^ibl(L'); 

— Find an estimate X of the shortest lattice vector length Xi{L') such that, 



ML') 

7VC2 



<A< Ai(T'); 



— Find the unique shortest vector ±v of L' , if L' has an -unique shortest 
vector; and 
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— Find a basis 6i, 62, . . . , such that the maximum length maxi<i<Ar 11^.11 < 

fv^bi(L'); 

where ci, 02,03 and 0 are absolute constants. 

4 NP-Hardness 

It was known that SVP, under the ^oo-norm, is NP-hard [47,66]. It was also 
shown there that the related Closest Vector Problem (CVP) is NP-hard for all 
Ip-norms, p>\. Arora et al. [5] showed that, under any Ip-norm, CVP is NP-hard 
to approximate within any constant factor, and that if it can be approximated 
within a factor of 2 *°s ^ then NP is in quasi-polynomial time. 

It had long been thought that the Shortest Vector Problem for the natural 
l 2 -norm is NP-hard. This was conjectured e.g., by Lovasz [52]. It remained a 
major open problem until, in 1997, Ajtai [2] proved the NP-hardness of the SVP 
for this norm, under randomized reductions. Moreover, Ajtai showed that to 
approximate the shortest vector of an n-dimensional lattice within a factor of 
(^1 + (for a sufficiently large constant k) is also NP-hard under randomized 
reductions. This was improved to (l -I- for any constant e > 0 by Cai and 
Nerurkar [20] , and then to any constant smaller than -\/2 by Micciancio [55] . 

Theorem 5 . It is NP-hard, under randomized polynomial time reductions, to 
find a shortest lattice vector, even to approximate it within a factor of -\/2 — e, 
for any e > 0. 

In the next subsection we outline Ajtai’s result. The presentation incorporates 
the simplifications and improvements of [20] but the main ideas are due to Ajtai. 
After that we present Micciancio’s improvement. 



Ajtai’s Result 

Ajtai gave a randomized reduction from the following variant of the subset sum 
problem to SVP. 

The Restricted Subset Sum Problem. Given integers oi, . . . , o/. A, each of 
bit-length < 1^, find a 0-1 solution to the system X)i=i ^ = 

LlJ. 

We first define a lattice which will play a crucial role in the proof. This lattice 
is a modified version of the one used by Adleman (unpublished) in his reduction 
from factoring to the SVP, under some unproven assumptions. For this lattice, 
we need to choose several parameters depending on the I in the restricted subset 
sum instance. 

— n is chosen to be a sufficiently large polynomial in I. 

— m is chosen to be a sufficiently large polynomial in n. 
m ^ n ^ 1 . 
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— 5 is chosen randomly from the set of products of n distinct elements of 
{pi, . . . ,Pm}, the first m primes. 

— w is chosen a constant root of b. 

— B is polynomial in uj. 

Clearly, B, b and oj are exponential in n. We will not be overly precise here 
about the values of these parameters in order not to obscure the main points. 
Using these parameters, Ajtai defines the following matrix, whose m+2 columns 
generate a lattice. 

/ ^/Ibgpl • • • 0 0 0 \ 



0 • • • VlogPm 0 0 

0 ••• 0 0 

\B logpi • • • B log Pm Blogb B log (l + f )/ 

Lattice La 

This lattice is then normalized. The normalized lattice has every vector of length 
at least 1 and a lot of vectors of length very close to 1. We will denote by Vi, 
the columns of the basis matrix for this modified lattice. We will denote this 
normalized matrix, as well as the lattice it generates, by L. With high probability, 
this lattice, L = L{v \, . . . , Vm+ 2 ), has the interesting properties we outline next. 
These properties are a consequence of the way primes are distributed and the 
convexity of the logarithm function. 

1. All non-zero vectors have length at least 1. 

2. There are a lot of vectors of small norm with the property that their first 

m basis coefficients G {0,-1}. More precisely, let Y be the set of all v G L, 
^ = Yh=i 'HT=i l“il = n, ai e {0, -1} for z G m}, and 

||z;|P < 1 -I- (5. Then |Y| > Here, <5 is an exponentially small quantity. 

3. Any two distinct elements of Y differ in their first m basis coefficients. 

4. If u is a non-zero vector of L of squared norm less than 1+ , then the first 

m + 1 coefficients of v have a special form. More precisely, if u = oavi, 

||w|P < 1 + and Um+i > 0, then ai, . . . , G {0, -1} and Um+i = 1. 

This lattice is now extended in the following random manner depending on 
the given instance of the restricted subset sum problem. With high probability, 
given a reasonably short vector in this extended lattice, a solution to the instance 
can be produced. 

Let = A be the given instance of the restricted subset sum prob- 

lem. Let e > 0 be any constant. Let r = 2/m*^ and jS = ^/t. Let C = Ci, . . . , C/ 
be a random sequence of pairwise disjoint subsets of {!,... ,z?z.}. Define an 
(^-1-2) X (m-|-2) matrix D as follows. The (m -I- 2)"'^ column is all zeros. The 
(m -I- I)*** column is {Alfi, 0, . . . , 0)^. The other entries of the matrix are 

defined in the following manner. 

1. The first row has the entry ailf3 in the j**' position if j G Ci, and otherwise 
has zero. 




The Complexity of Some Lattice Problems 



15 



2. The second row has the entry 1(3 in the position if j is in some Ci, and 
otherwise has zero. 

3. For i from 3 to ^ + 2, row i has (3 in the position if j G Ci -2 and otherwise 
has zero. 

If C\, . . . ,Ci are consecutive intervals of m}, then D is the following 

matrix, 



/a\l(3- ■ ■a\l(3- ■ ■ail(3- ■ ■ail(3- ■ ■ Al(3 0\ 
1(3 ■■■ 1(3 ■■■ 1(3 ■■■ 1(3 ■■■[^\l(3o\ 

(3 ■■■ P ■■■ 0 ■■■ 0 ■■■ 0 0 



\ 0 ■■■ 0 ■■■ P ■■■ P ■■■ 0 0 / 



The extended lattice is the lattice generated by the columns of the matrix 



. A vector v € can be written 



, where for some integral column 



vector a = (oi, . . . , am+ 2 )’^ , v = ^ ^ ~ Each v uniquely 

determines its ct and thus uniquely determines v' . 

Ajtai uses a constructive variant of the following combinatorial lemma, due 
to Sauer, to show that any solution to a subset sum instance can be produced 
from the coefficients of some short vector. A proof of this lemma can be found, 
for example, in [4] . 



Lemma 3 (Sauer). Let S he a finite set and S he a set of subsets of S. If 
for some k, |5| > i^T)’ then there is a X C S with k elements such that 

2^ = {XnZ \ Z €S}. 



That is, every subset of X can be realized by intersecting it with some element 
of S. A consequence of Ajtai’s constructive lemma is that a random sequence 
C = Cl, . . .Cl of subsets of {1, ... , m}, has the following property: 



Vs G {0,1}^ “jUj G Y such that, Vz G {I,-- - ,l},Si = 

~ ^0- 

This property implies that if there is a solution to the restricted subset sum 
instance then there is a vector in the set Y that gives rise to it. That is, suppose 
Y^i=i ^ has a solution Xi = Si, i.e. 



i i 

Si G {0,1}, '^aiSi = A and ^ Si 

i=l i=l 



L-J 



Then, G T, v = “jUj, such that Vz G (1, . . . , 1}, 






= -E 



jeCi 
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Since f G Y, 0 < ||w|p < 1 + <5. Let v G L^^\v = 
a = (ai, . . . , ara+ 2 )'^ ■ Let v' = {v'l, , v[_^ 2 )- Then 



where v' = Dot and 



ll^f = lkf + lk'f <(l+<5) + rL^J <l + rL (3) 

The first inequality holds because v'l = v '2 = 0, and exactly [|j of u' for z > 3 
are —(3, the rest being zero. The last inequality holds because 5 is exponentially 
small. Also, since u is a non-zero vector, so is v, which implies 

Ai(T(^))<||fl||. (4) 



We now prove that, assuming a solution to the restricted subset sum instance 
exists, one such solution can be constructed from an approximate shortest vector. 



Let w = 



be a (1 -|- 0 approximate shortest non-zero vector of i.e. 



<(l + 0Ai(L(^)r. (5) 

We will construct a solution to the subset sum instance, given w. Since r = 2/m*^, 
this shows that it is NP-hard to approximate the shortest vector within a factor 
constant e > 0, where dim stands for the dimension of the 

lattice. 

From (3), (4) and (5) we get 

Ikf < (l+0(l + rO, (6) 



and by the choice of r and m 1), one can show that 



llzhf <1 + 



2 

^ 3£/4 



This matches the bound in property 4 of L. Let w = (wi, . . . ,Wm+ 2 ), w' = 
{w'l , . . . , ZC/_|_ 2 ) and w = By property 4, replacing w by —w if nec- 

essary, 7 m+i = 1. We now prove that 



Vi = ~Y^ 7j 

3&Ci 

is also a solution by showing that, if not, the length of w would be too large. It 
is easy to see that since 7m-i-i = 1> 



i 

w[ = PI{A - y^azz/z), 
w '2 = 
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and for 1 < j < I, 



Wj+2 = -Pyj- (J) 

Assume the yi are not a solution. Then, at least one of the following three 
conditions must hold. 

1) ELi ^ A or 

2) Ei=l Vi 7^ L 2 J > 

3) 3i yi ^ {0, 1}. 

If 1) holds, then > pi, which means 

\\w\\^ = \\yjr+\\yjT>l + f3^l^ = l + n\ 



where ||tc|| > 1 holds by property 1 of L. This contradicts (6). If 2) holds, then 
1^2! ^ we get a similar contradiction again. Finally, it can be shown that 

if for some i, yi ^ {0, 1} and Ej=i Vj = L|J> then 



i 






1 

2 



+ 2. 



This means, by (7) and property 1 of L, 



lltZif = ||tcf + ||tc'f >l + r 



1 

2 




Since ||u|p < (1 + (5) + r[|J (see (3)), 

INII^ ~ l|tl|P > 2t — S > T. 



Due to our choice of m as a sufficiently large polynomial in we have 

tI = — I < 1. 



Thus by (3), ||t>|p < 2, and so 




Therefore, 

INf >(1 + 0 llilf >(1 + 0 Ai(E^)r, 

which contradicts (5). 

This completes the proof of Ajtai’s result. 
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Micciancio’s Improvement 

With the same basic framework, but using the closest vector problem instead 
of the restricted subset sum problem, Micciancio [55] got an improved hardness 
result for the SVP. He showed that it is NP-hard, under randomized reductions, 
to approximate the SVP to within any constant smaller than -\/2, using the fact 
that it is NP-hard to approximate the CVP to within any constant. (In fact, it 
is even NP-hard to do so to within a factor 2*°® ”, for an e = o(l) [26], but 

this does not seem to lead to any improvement in his proof.) 

To describe this result, it is convenient to formalize the approximation prob- 
lems as promise problems [29] . The following defines the problem to approximate 
the closest vector within a factor c > 1. 

CVP Promise Problem 

Given an instance (H, y, d), where B G is a basis matrix, y G Z” is 
a target vector, and d G R, with the promise that either \\Bx — y|| < d 
for some x G or \\Bx — y|| > cd for all x G Z^, decide which is the 
case. 

Arora et.al.[5] showed that for all constants c > 1, this promise problem is NP- 
hard. From the proof in [5] one gets that even the following modified version of 
the above problem is NP-hard for all constants c > 1 . 

Modified CVP Promise Problem 

Given an instance {B,y,d), where B G Z”^^, y G Z”, and d G R, 
with the promise that either \\Bx — y\\ < d for some x G {0,1}^, or 
\\Bx — ay\\ > cd for all x gZ^ and for all a G Z \ {0}, decide which is 
the case. 

We will call instances that satisfy the first alternative, YES instances, and those 
that satisfy the second one, NO instances. Note that, in the modified problem, 
a YES instance has a 0-1 solution and a NO instance has no solution even for 
arbitrary integral x and arbitrary (non-zero) multiples of the target vector. 

Here is the definition of the corresponding SVP promise problem. It formal- 
izes the problem of approximating the SVP within a factor c' . 

SVP Promise Problem 

Given an instance (V) t), where V is a basis matrix, and t gR, with the 
promise that either |lVw|l < t for some non-zero integral w, or ||Vw|| > 
cT for all non-zero integral w, decide which is the case. 

We define YES and NO instances in a similar manner. 

Micciancio gave a randomized many-one reduction that reduces the modified 
GVP promise problem with c = y^2/e to the SVP promise problem with c' = 
a/ 2/(1 -I- 2e), for any constant e > 0, mapping YES instances to YES instances 
and NO instances to NO instances. This shows that the SVP is NP-hard to 
approximate within any constant smaller than \/2. 

The heart of his proof is a technical lemma that asserts the existence of 
a probabilistic algorithm that on input 1^, where k is from the GVP promise 
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problem instance, constructs a lattice L G a matrix C £ and 

an s G , such that with high probability, 

— For every non-zero 2 G Z™, ||Lz|P > 2, and 

— For all X G {0, 1}^, 3z G Z™, such that Cz = x and \\Lz — sW^ < 1-1- e. 



Here, m depends polynomially on k. 

The lattice L above is essentially the same as Ajtai’s lattice La and C can 
be thought of as representing the 0-1 vector x by 2 . The existence of such a C 
and the fact that such a C can be randomly constructed depends on a version 
of Sauer’s Lemma. 

Let {B, y, d) be a given instance to the CVP promise problem with c = i/2/e. 
The reduction maps it to the instance {V, t) of the SVP promise problem with 
c' = a/2/(1 -I- 2e), where 



V = 



(^BC 




and t = \/l + 2e. Note that c't = \/2. 

Let {B, y, d) be a YES instance. That is, \\Bx — y\\ < d for some x G {0, 1}^. 
Then 3z G Z™, such that \\{BC)z — y\\ < d and ||L 2 — s|p < 1 -I- e. Let w be the 

vector . Then 



\\Vwf <{l + e) + ^-d‘^ = l + 2e = f. 



Let {B, y, d) be a NO instance. Let w = 
where z G Z'" and a G Z. If a = 0, then z yf 0 and so 

llPwll > ||Lz|| > V 2 = c't. 



be a non-zero vector in 



If a yf 0, then 



lly^ll > ^\\B{Cz) 




d = V 2 = c't . 



This completes the description of Micciancio’s result. 



Other Hardness Results 

Dinur, Kindler and Safra [26] have recently improved the hardness factor for 
CVP. They show that CVP is NP-hard to approximate within a factor 2 *°s 
for an e = o(l). Blomer and Seifert [10] study two problems considered by Ajtai 
in his worst-case/ average-case connection. These are the problems of computing 
a shortest set of independent lattice vectors and a shortest basis. Using the result 
of [26], they prove that both these problems are hard to approximate within a 
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factor for some constant c < 1. Goldreich et al [36] show a reduction 

from the CVP to the SVP. While this reduction does not give us an improved 
hardness result, it has the properties of preserving the factor of approximation 
for the two problems and the dimension of the lattice. 

Ravikumar and Sivakumar [61] consider the problem of deciding whether a 
lattice vector shorter than a given bound exists, under the promise that there is 
at most one such vector (not counting its negation). They prove a randomized 
reduction from the decision version of the general shortest vector problem to this 
problem, in the style of Valiant and Vazirani [64]. Lattice problems for a special 
kind of lattice defined by certain graphs have been studied in [18]. 

5 Non-NP-Hardness Results 

To what extent can we expect to improve further the approximation factor for 
SVP and remain NP-hard? The current proof appears not feasible beyond \/2. 
On the other hand, the best polynomial time approximation algorithms of Lovasz 
and Schnorr are exponential in the approximation factor. 

For polynomially bounded factors, transference theorems provide evidence 
that beyond a factor of 0(n), the approximate SVP is not NP-hard. This is 
a result of Lagarias, Lenstra and Schnorr [48]. Transference theorems in the 
Geometry of Numbers give bounds to quantities such Ai of the primal and the 
dual lattice. In [48] the following theorem is proved 

1 < Ai(L)A„_i+i(L*) < —n^, 

6 

for n > 7,1 < i < n. This already gives an “NP proof” for a lower bound 
for Ai(L) up to a factor of 6>(n^) by guessing an appropriate set of linearly 
independent lattice vectors of L* all with length at most A„(L*). 

Lagarias, Lenstra and Schnorr [48] proved more. A basis {6i, 62 , . . . , bn} is 
said to be reduced in the sense of Korkin and Zolotarev, if the following hold: 

1. ||6i|| = Ai(L). ^ 

2. Let {61, 62, , bn} be the Gram-Schmidt orthogonalization of {^i, . . . , bn}j 

bi — bi ^ ^ f^ikbki 1 ^ ^ ^ 
k<i 

Then < 1/2, 1 < k < i < n. 

3. If is the orthogonal projection of L to (lsp{6 i,... then 

||6,|| = Ai(L("-*+D). 

Essentially, a Korkin-Zolotarev basis is one which is weakly reduced, and the 
orthogonal projection of bi is a vector of minimum length in the orthogonal pro- 
jection of L in the complement of {bi, . . . , bi-i}. In terms of Lovasz’s algorithm, 
if instead of comparing bi{i) and 6i+i(z), we searched for a vector of minimum 
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length in lsp{6i(z), . . . , 5„(i)}, and called it bi, we would have obtained a Korkin- 
Zolotarev basis. (Of course then this algorithm would have run in exponential 
time.) 

Let B* be a Korkin-Zolotarev basis of L* . Its dual basis B = {bi, 62 , . . . , bn} 
is called a dual Korkin-Zolotarev basis of L. Let X{B) = min{||6i|| | 1 < z < n}, 
where {61,62 , . . . , 6„} is the Gram-Schmidt orthogonalization of B. Then it is 
shown in [48] that 



\{B) < Ai(L) < nX{B). 

In particular this gives a way to provide an “NP proof” of a lower bound 
for Ai(L) up to a factor of n by guessing an appropriate basis B* and then 
calculating B. This places the promise problem of approximating Ai(L) up to a 
factor n within coNP.^ Thus if NP yf coNP, then approximating Ai(L) up to a 
factor n is not NP-hard in the sense of Karp reductions. More precisely, if NP 
yf coNP, then there is no deterministic polynomial time reduction a from SAT, 
= (L, A), such that if G SAT, then Ai(L) < A, and if ^ SAT, then 
Ai(L) > nX. 

Theorem 6 (Lagarias, Lenstra, Schnorr). If NP yf coNP, then the prob- 
lem of approximating Ai(L) within a factor n is not NP-hard. 

The interplay between the primal and dual lattices and the related transfer- 
ence theorems play important roles in Ajtai’s worst-case to average-case connec- 
tion as well. We will discuss this topic in more detail in the next section. Here 
we present the following rather pretty result due to Goldreich and Goldwasser 
which improved the approximation factor for non-NP-hardness to ^/n. 

The proof of Goldreich and Goldwasser [32] is based on constant round in- 
teractive proof systems. More precisely, they give a bounded round interactive 
proof system for proving a lower bound up to a factor for both SVP as well as 
GVP. Of course the number of rounds can be reduced to one, either by standard 
techniques or by directly parallelizing their IP protocol. Also by standard tech- 
niques private coins can be replaced by public coins, so that what they showed 
can be stated as follows: 

Theorem 7 (Goldreich, Goldwasser). The problem of approximating Xi{L) 
within a factor ^/n is in NP n coAM. Thus if this problem is NP-hard under 
Karp reductions in the sense given above, then = iT|. 

The last statement follows from a well-known result of Bopanna et. al. [12] 
which states that if coNP C AM, then = iT|. 

The restriction to Karp reductions has been improved recently to general 
Gook reductions to promise problems in this result [22] . 

^ Of course, technically a promise problem is not a decision problem while coNP is a 
decision problem class. But the meaning of this is clear and one can always modify 
the definitions slightly to make it proper. 
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The basic idea of the IP protocol of [32] is rather simple and elegant and we 
will describe it here. 

Suppose L satisfies the promise of either \\{L) < t or Xi{L) > and the 

prover claims that Ai(L) > t ■ ^/n. Imagine we surround each lattice point p € L 
a ball Bp{r) centered at p with radius r = t ■ ^/nj2. If the prover P is correct, 
then all such balls are disjoint. Now the verifier randomly picks a lattice point 
p in secret, and randomly picks a point z in Bp(r). The verifier presents z to 
the prover, who should respond with p, the center of the ball from which z was 
chosen. It is clear that for an honest prover P with unlimited computing power, 
since all the balls Bp{r) are disjoint, he has no difficulty meeting his obligation. 
However, suppose the prover P' is dishonest, so that in fact Ai(L) < t. Then for 
any lattice point p picked by the verifier, there is at least one nearby lattice point 
p' with jjp — p'jj < t. Then Bp{r) and Bpi(r) would have a large intersection. 
This follows from the fact that the radius is almost times the distance of 
their respective centers. It follows that there is a significant probability that a 
dishonest prover will be caught, since in case a point z G Bp{r)(^Bpi{r) is chosen, 
the verifier could equally have chosen p or p' . 

The exponent 1 /2 in this interactive proof protocol comes from the well 
known fact that in n-dimensional space, two unit balls with center distance 
d have a significant intersection if d < 1/ ^/n, and a negligible intersection if 
d > for any e > 0. With some care the proof in [32] can improve the 

factor y/n to i/n/logn. It also shows the same bound for the Closest Vector 
Problem. 

What about some other problems? The problem of n'^-unique shortest vec- 
tor problem is prominent in the Ajtai worst-case to average-case connection. It 
also plays an important role in the Ajtai-Dwork public-key cryptosystem. Recall 
that a lattice is said to have an n°-unique shortest vector if \ 2 {L)/\i{L) > rf. 
Equivalently, there exists v G L, v ^ 0, such that for all v' G L, if | |w'| | < 1 1 1, 

then v' is an integral multiple of v. 

Define the following promise problem: 

The n°-Unique Shortest Lattice Vector Problem: 

Given a lattice with a n°-unique shortest vector v, find the shortest vector ±w. 

Building on the idea of Goldreich and Goldwasser [32], Cai [16] proved the 
following: 

Theorem 8. The rd^-unique shortest lattice vector problem for c < 1/4 is not 
NP-hard under Karp reductions unless the polynomial time hierarchy collapses 
to IJP = iTf. 

It is not yet clear whether Theorem 8 can be improved to hold for general 
Cook reductions as well. 

6 Transference Theorems 

We have already mentioned the transference theorem of Lagarias, Lenstra and 
Schnorr [48] in the last section. There is a long history in geometry of numbers 
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to study relationships between various quantities such as the successive minima 
associated with the primal and dual lattices, L and L* . Such theorems are called 
transference theorems. The estimate for the product 

K{L)\n-i+i{L*) 

has a illustrious history: Mahler [54] proved that the upper bound (n!)^ holds 
for all lattices. This was improved by Cassels [24] to n\. The first polynomial 
upper bound was obtained by Lagarias, Lenstra and Schnorr [48] as mentioned. 
The best estimate for this product is due to Banaszczyk [9], who showed that 

1 < A,(T)A„_i+i(T*) < Cn, 

for some universal constant C . The Banaszczyk bound is optimal up to a con- 
stant, for Conway and Thompson (see [56]) showed that there exists a self-dual 
lattice family {T„} with Ai(L„) = Q{\/n). 

Part 2) and part 3) of Ajtai’s worst-case to average-case connection in The- 
orem 3 are proved via transference type argument. Basically, if one can get a 
good estimate for the basis length for any lattice, one can apply this to the 
dual L* . From a good estimate for bl(L*), thus A„(L*), a transference theorem 
gives estimate for Xi{L). This is part 2) in Theorem 3. Part 3) employs some 
additional argument also of a transference type. We will discuss these matters 
in more detail. But first we take a closer look at transference theorems. 

In addition to Aj, there are several other lattice quantities that have been 
studied. The covering radius of L is defined to be the minimum radius of balls 
centered at each lattice point whose union covers M”. 

/i(T) = minjr | L + i?(0; r) = M”}. 

Also if d(u, L) denotes the minimum distance from a point u in M" to a point in 
L, then 



fi{L) = ma,x{d{u, L) \ u G M"}. 

(The minimum and maximum are obvoiusly achieved.) 

We have seen the quantity 

^ = sup max Ai(L)A„_i+i(L*), 

L 

where the supremum is taken over all n-dimensional lattices. Regarding covering 
radius /i(T) the relevant quantity is 

7 ] = sup^(L)Ai(T*). 

L 

By triangle inequality fJ.{L) < Xn\n{L), so that 
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Given any L, we say a sublattice L' C L is a saturated sublattice if L' = LOU, 
where II is the linear subspace of M” spanned by L' . Saturated sublattices of 
dimension n — 1 are in one-to-one correspondence with primitive vectors of L* . 
(A lattice vector u yf 0 is primitive if it is not an integral multiple of any other 
vector in the lattice except ±u.) The correspondence is simply L' = If] {u}''" 
and {u}-*- = Isp(T'). For any L and a saturated sublattice L' of dimension n—1 
with normal (and primitive) vector v S L*, L is a, disjoint union of parallel 
translations of L' , 

L = [J (A' -I- ku), 
fcez 

for some u £ L such that (u, u) = 1. Thus, each pair of nearest hyperplanes 
{«}■*■ -I- ku and {u}-*- -I- (fc -I- l)u has orthogonal distance {u, jj^) = We call 
this a parallel decomposition of L. 

For any L and any u G R”\T, we can compare d{u, L), to the distance from 
u to the closest parallel translation of some {u}-*- = Isp(L') which intersects L, 
over all such L' . Let 



{u,v}) = \{u,v) - r(u,u)J| 
be the fractional part of (u, v) rounded to the nearest integer, then we consider 

dz{{u,v)) 



S = sup 

v^L* , {u,v) 



which measures the distance from u to the closest parallel translation, maximized 
among all directions v € L*. Now the following quantity is defined 

d{u,L) 

C = sup sup — y — . 

L u6R"\L a 

By definition dz{{u,v)) < 1/2 and ||u|| > Ai(T*), so that 5 < ■ Hence 

C>2r?. 



An upper bound (^ < (3 says that VT and Vu ^ L, there exists a parallel decompo- 
sition where the distance from u to the nearest lattice hyperplane is > Pd{u, L). 

Lagarias et al. [48] proved that ^ and 77 < ■ Babai [ 6 ] proved that 

C < C” for some universal constant C . Hastad [40] showed that C < -I- 1. 

Similar bounds for 77 and C were also shown by Banaszczyk [ 8 ]. The best 
bounds for 77 and C were shown later by Banaszczyk [9], where 77 and C are 
all bounded by 0(n). The Banaszczyk bounds are all optimal up to a constant 
by the Conway-Thompson family of lattices (see [56]). 

In [14] an extension of Banaszczyk’s theorem of [9] is proved. Define gi(L) to 
be the minimum r such that the sublattice generated by L H B(0; r) contains an 
i-dimensional saturated sublattice L' , where 1 < i < n. When z = n, it is called 
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the generating radius and is denoted by g{L). Clearly g{L) is the minimum r such 
that a ball B{0; r) centered at 0 with radius r contains a set of lattice vectors 
generating L. The study of g{L) is motivated by the investigation of bl(L) and 
its relation to A„(L). Clearly 

A„(T) < g{L) < bl(L). 

The following inequality is shown in [14] for every lattice L of dimension n, 
using and extending the techniques of [9]: 

5 ,(L)-A„_i+i(T*)<Cn, (8) 

for some universal constant C, and for all z, 1 < z < n. We will sketch the proof 
for the case i = n for the generating radius g{L). 

The main tools of the proof are Gaussian-like measures on a lattice, and their 
Fourier transforms. For a given lattice L we define 

-^({-}) = ^ .--IN, - ( 9 ) 

The Fourier transform of ctl is 




(10) 

( 11 ) 

(12) 



(Jl{u) = (13) 

The proof of Lemma 4 uses Poisson summation formula, see [42,9] . The following 
lemma is proved in [9] and is crucial: 

Lemma 5. For each c > l/-\/27r, 

aL{L\B{Q]Cy/n)) < (c\/2TTee~^‘^ ^ , (14) 

and for all u G M”, 






where i?(0; ci-^/zz) is the n-dimensional hall of radius ci^/n centered at 0. 
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This lemma basically says that the total weight under ctl of all lattice (or affine 
lattice) points outside of radius C\/n is exponentially small. 

Now we prove (8) for z = n and C = 3/(27 t). Suppose g{L)Xi{L*) > inj2n. 
Let Cl and C2 be two constants, such that C1C2 > 3/27 t and ci > Ij^/^ and 
C2 > 3/-\/27r. By substituting L with sL for a suitable scaling factor s, we may 
assume that 

g{L) > ci^/n and Ai(L*) > C 2 \/n. 

Let L' be the sublattice of L generated by the intersection L n B{ 0 ; c\^/n). 
Then L' is a proper sublattice of L, since g{L) > ci^/n. If dimL' < n, then 
let P be the linear span of L', and let bi, . . . ,bi be a lattice basis of L n P, 
where z = dim L' < n. This can be extended to a lattice basis bi, . . . ,bi, . . . , 
for L and we may replace L' by the sublattice generated by 61, . . . , , 26„, 

say. Thus without loss of generality we may assume L' is of dimension n. The 
important point is that we have a proper sublattice L' C L, which is of dimension 
n and contains L n P(0; ci^/n). 

For any fixed u G M”, 

ox(u) = ^ ctl({v}) cos(2tt(u, v}) 

vGL 

= X! <^L'({f})cos(27r(zz, u)) 

vGL' 

+ “ ^ L '{{ v })) cos( 27 t(zz, z;)) 

vGL’ 

+ ^ (TL({z;})cos(27r(zz, u)) 

vGL\L' 

= ^i{u) + A + B, say. 

Since L n B{ 0 ; c\^/n) C L' , the last term 

\B\ < ^ CfL{{v}) 

v^L\B{ 0 \c\y/ri) 




by Lemma 5 inequality (14). Denote the last term by e”, say. 

For the other error term A, we can show similarly that 

1^1 <e^ 

Hence 

ctl(zz) > (7l/(zz) - 2e” . (16) 

Our next task is to show that we can choose an appropriate u so that o^{u) 
is small yet is large. By Lemma 4, we have (Tl(u) = and = 




The Complexity of Some Lattice Problems 



27 



'T(l')* (u). Thus we only need to choose a u such that tl* (u) is small and T(l')* (u) 
is large. 

The following lemma is proved in [14]. 

Lemma 6. Suppose L\ is a proper sublattice of L 2 , then there exists a p G L 2 , 
such that 

min \\p-q\\> 

qGLi 6 



(Since a lattice is a discrete subset of M”, the above minimum over q clearly 
exists.) 

Now we note that since L' is a full ranked proper sublattice of L, L* is a 
proper sublattice of {L')*. That it is proper follows from the identity of index 



det((T')*)/det(L*) = det(T)/det(L') > 1 . 

By Lemma 6 , take a u G (L')*, such that mimei. Ilu— oil > ^4^. Then since 
u G {L'Y, we have {L')* + u= (L')% and 



T(i,).(u) = 



E 



xG{L')* -\-u * 






= 1 . 






On the other hand, since 



• II II \ ^ C2 ^ 

min jju- 9|| > > —Vn, 

qeL* 0 6 



we note that no point in L* + u is within in norm, and so 






V p-’^lkll 

A^x^L* -\-u ^ 

E.eL*e-lldl^ 



2 



< 2 




2 c 2 say. 



by Lemma 5 inequality (15). Since both ci and 02/8 > l/-\/27r, we have both ei 
and C 2 < 1 by elementary estimate. Thus it follows from (16) that 



2e^ > 1 - 2e^ 



which is a contradiction for large n. 

For the special class of lattices possessing n*^-unique shortest vector, a stronger 
bound is proved [15], which lead to a further improvement in the Ajtai connection 
factors of part 2) and 3) in Theorem 3. 

Theorem 9. For every lattice L of dimension n, if L* has an n'^-unique shortest 
vector, then 



1 < A„(L)Ai(L*) < 0(n‘5), 
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where 



{ 1-c z/0<c<l/2, 

1/2 z/l/2<c<l, 

3/2-cz/l < c< 3/2, 

0 if OS/2. 

In terms of the Ajtai connection factors in Theorem 3 — in part 2) and part 
3) — these new transference theorems improve all the factors to the range of 
approximately 3 and 4. Details can be found in [15]. Here we outline the general 
idea to derive parts 2) and 3) from 1). 

The idea for the estimation of Ai(T) is relatively straightforward. From an 
estimate of the maximum length of a set of linearly independent vectors from 
L*, one gets an estimate of Xi{L), via transference theorem. 

To actually compute the shortest vector, the following idea is due to Ajtai [1] . 
If L* has an n'^-unique shortest vector v, then L admits a parallel decomposition 

L = [J + ku), 
fcez 

where the parallel hyperplanes containing L' + ku have orthogonal distance much 
larger than the basis length of L' . Now randomly sample a large polynomial 
number of lattice points within a certain bound. A l/n*^*-^^ fraction of samples 
fall on the same parallel hyperplane, and the difference vector of such a pair 
belongs to the hyperplane Isp(L'). If we can distinguish such pairs from the rest, 
then we can identify the normal vector for the hyperplane Isp(L'), and by taking 
out the gcd, we can recover the shortest vector ±v. 

For two sample lattice points x and y, if they belong to the same parallel 
hyperplane, then by including a small fractional vector (x — y) /N to the gener- 
ating set of L, one does not change bl(T), since this is controlled by the distance 
between the parallel hyperplanes. 

But if X and y belong to different parallel hyperplanes, then by including 
{x — y)/N to the generating set of L, the new lattice will have many additional 
parallel translations of L' between any two originally adjacent parallel hyper- 
planes Isp(T') -I- ku and Isp(L') -I- (fc — l)u. This will reduce the basis length 
significantly. 

Thus to be able to compute a good estimate of the basis length for L (ac- 
tually an estimate of A„(L) will do) leads to the identification of the unique 
shortest vector for L*. Clearly improved transference theorem bounds sharpen 
the provable estimates in Ajtai’s worst-case to average-case connection factors. 
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Abstract. We give a new algorithm using hnear approximation and lat- 
tice reduction to efficiently calculate all rational points of small height 
near a given plane curve C. For instance, when C is the Fermat cubic, we 
fold aU integer solutions of \x^ -\- tf' — z^\ < M with 0<x<y<z<N 
in heuristic time ^ (log®*-'"^ W)M provided M ^ N , using only 0(log N) 
space. Since the number of solutions should be asymptotically propor- 
tional to MlogW (as long as M < A®), the computational costs are es- 
sentially as low as possible. Moreover the algorithm readily parallehzes. 
It not only yields new numerical examples but leads to theoretical re- 
sults, difficult open questions, and natural generahzations. We also adapt 
our algorithm to investigate Hall’s conjecture: we find all integer solu- 
tions of 0 < — j/^l ^ x^^^ with r < A in time log®*-'"^ A). By 

implementing this algorithm with A = lO'"® we shattered the previous 
record for — y^\. The 0(A''^^ log®*-'"^ A) bound is rigorous; its 

proof also yields new estimates on the distribution mod 1 of for 

any positive rational c. 



1 Introduction 

One intriguing class of Diophantine problem concerns small values of homo- 
geneous polynomials. In the simplest nontrivial case of a polynomial in three 
variables defining a projective plane curve C : P{X, Y, Z) — 0, the problem can 
be reformulated thus: given a plane curve C, describe for each positive N, S the 
rational points of height at most N in which are at distance at most J of C. 
With present-day methods, hardly any nontrivial results can be proved on the 
number or existence of such points. But one can still seek numerical evidence, 
and ellicient algorithms for obtaining this evidence. The direct approach is to try 
all X, y with \x\, \y\ < N, and for each pair to solve P{x, y,z) — 0 for ^ £ [~N, A], 
recording those cases in which ^ is sulliciently close to an integer. This requires 
space 0(log(A)) but time (A^ -|- JA®) log*^*-^^ A, which is inellicient once S is 
much smaller than A“^ since for general C,N,S ^ N~^ the number of solu- 
tions should be proportional to SN^. We give a new algorithm, also requiring 
only 0(log(A)) space, but with heuristic running time [N+SN^) log*^^^^ A. Thus 
as long as J N~^ we expect to find all the points of height < A and distance 
< J in time only log'^^^^A per point. Moreover, our method readily parallelizes, 
since it divides the computation into many independent subproblems. 
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We describe this algorithm, give the heuristic estimate for its run time, and 
briefly discuss the problem, which seems quite dilhcult, of proving our heuristic 
time estimates. We prove (Thm.l) that an alternative description of those points 
can always be computed in the heuristically expected time. We then discuss 
natural generalizations to other valuations and higher dimensions. 

An algorithm for Ending rational points near a variety can in particular 
find rational points on the variety; applying our methods to embeddings of the 
variety in projective spaces of high dimension we obtain a new approach to 
this fundamental problem in computational number theory which improves on 
existing methods in several important cases. This approach also works for non- 
algebraic varieties, and even yields a theoretical result (Thm.4) on the paucity 
of rational points on non-algebraic analytic curves. 

We next describe experimental results of the implementation of our algorithm 
to various curves of interest, notably the Fermat curves of degree n > 2, where 
some of our experimental findings led us to new polynomial families of small 
values of — J/" — x^\ (Thm.5). We devote a separate section to the case 
of the cubic Fermat curve, corresponding to small values of \z^ — 'tf' — x^\, a 
problem for which there is already some literature and the heuristics are subtler. 
In particular, we found for several integers d < 10® the first representation of d 
as a sum of three integer cubes; D.J. Bernstein has since extended the search up 
to A = 2 • 10® and beyond, and found many new solutions, including one for 
d = 30 which was a long-standing open problem. 

Finally we show how to modify our algorithm to elhciently search for small 
nonzero values of \x^ — J/®|- This is the topic of Hall’s conjecture, which is part 
of a web of important Diophantine problems surrounding the ABC conjecture 
of Masser and Oesterle. The conjecture asserts that x^ — j/® is either zero or 
x^l’^~‘'- for all x,y We are able to find all solutions of 0 < \x^ — y^ \ ^ x^!"^ 
with X < X m time X), again using only (9 (log A) space. Using 

this improvement on the obvious Alog'^^^^^A method of trying all a? < X, we 
computed all cases of 0 < \x^ — y^\ < x^!"^ with X < 10^®. We found ten new 
solutions, including most notably 

5853886516781223® - 447884928428402042307918® = 1641843 

with x^!"^ j\x^ — t/®| = 46.600-I-, improving the previous record by a factor of 
almost 10. In this case the time estimate is not heuristic; its proof not only 
streamlined the computation but even yields new theorems on the distribution 
mod 1 of [cx)^^^ for any positive rational c. We announce some of these results 
at the end of the present paper; the full statements and proofs will appear 
elsewhere. 
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2 The Algorithm in Theory 

2.1 Specification and Heuristic Analysis 

While we are mainly interested in algebraic plane curves C, the algorithm does 
not require so strong a hypothesis: we can hnd^ 2063’^ + — 8128’^ = 

0.019369- as well as 386692^ + 411413^ = (1 - 1-035 . . . • 10-^®)441849^. All 
we need is that C is the image of a differentiable map : [0,1] RP^ with 
bounded second derivatives. Fix a positive 5 < 1, and assume S N~^ for rea- 
sons given in the next paragraph. Partition [0, 1] into intervals Im each 

of length \Im\ — On each Im, approximate (f> to within 0{\lm\^) — 0{S) 

by a linear approximation <j>. Then a point at distance < 5 from <f>{Im) remains 
at distance ^ 5 from <f>{Im)- 

We now treat each Im independently. The triples [x, y, z) £ Z® — {0} such 
that [x \ y \ z) d has height < N and is within 0(5) of <f>{Im) are among 
the nonzero integer points in a parallelepiped Pm of height, length and width 
proportional to N,S^^^N,SN . Thus we expect that \Pm fl Z^| is approximately 
the volume of Pm, provided that this volume is 1. This is the case once 
5 5^ N~^. (That is why we insisted that 5 N~^: choosing smaller 5 would 

only make us work at least as hard to hnd fewer points.) Listing all the points in 
Pm n Z® is a standard application of lattice reduction. Let Mm be an invertible 
3x3 matrix such that Mm Pm is the cube K — [—1,1]®. We are then seeking 
all V £ Z® such that MmV £ K, or equivalently all vectors in K fl Mm^H^. We 
hnd them by reducing the lattice M“^Z®. This gives us a matrix Lm £ GL 3 (Z) 
such that MmLm is small. Now MmV £ K if and only if w £ Z®n {MmLm)~^K 
where v — LmW. But {MmLm}~^ K is contained in the box centered on the origin 
whose j-th side is twice the norm of the f-th row of {MmLm}~^ (* = 1, 2, 3). For 
each nonzero integral w in this box, calculate [x, y, z) — LmW and test whether 
[x \ y \ z) m. fact has height < N and lies within 5 of C . Doing this for each m 
yields the full list of such points. 

As advertised, the algorithm requires only O(logA) space (though much 
more space is usually needed to store the results of the computation). Also, 
since each of many intervals Im is treated independently, the computation can 
be massively parallelized with little loss among processors that interact only by 
reporting each [x \ y \ z) io headquarters as it is found. How long do we expect 
the computation to take? We assume that <f> and its derivatives can be calculated 
to within in time log*^^^^ A. Such is the case for all curves we consider and 

for every algebraic plane curve. Then each Mm takes only log'^^^^A operations 
to compute. Each lattice reduction can also be done in time polynomial in log A, 
since our lattices are in hxed dimension — and moreover our dimension of 3 is 
small enough that Minkowski reduction is described explicitly. [For an overview 
and further references concerning Minkowski reduction, see [CS, pp. 396-7].] So 

^ Our computations indicate that the first example is probably the smallest value of 
1*:^ P — z^\ for positive integers x, y, z, and at any rate the smallest with < 10®; 
and the second is the smallest ratio oi \x^ + y^ — z~^ \ to z~^ , and even to z*', for positive 
integers satisfying x < y <. z < 10®. See the next section. 
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far this amounts to ^ N time up to the usual log factors. Now each 

Pm. has volume 2®/|detMm| ^ If each {MmLm}~^ had all of its 

entries 0{S^^^N) — equivalently, if the shortest nonzero vectors of each lattice 
had length — then there would only be choices 

for w, which summed over m gives 0{SN^). Thus the total work would indeed 
be log'^^^^N times the expected number of solutions. Unfortunately it is too 
optimistic to expect that the entries of {MmLm}~^ are all ^ If the 

lattices are randomly distributed in the space of lattices of covolume 

in R®, some of them will have nonzero vectors much shorter than 
However, the average number of lattice vectors in K of a random 
lattice of determinant D is still 0{1/ D). Thus we expect — and typically hnd in 
practice — that, even accounting for the occasional short lattice vector, we will 
hnd all rational points of height < N that lie within S of C, doing on average 
log'^^^^N work per point. 



2.2 Can the Estimates Be Made Rigorous? 

Our assumption that the lattices are randomly distributed was not 

proved; indeed it is false at least for some choices of C. Most glaringly, if C 
is a rational straight line then there are rational points on C, and a for- 

tiori at least as many at distance < J. While we of course will not apply our 
algorithm to straight lines, we do apply it to the n-th Fermat curve, which has 
contact of order n with several rational lines such a,s y — z; each of those lines 
contains points at distance ^ 1/N^ from the curve, exceeding the 

expected count of N N once n > 2. (These are the points we exclude by 

imposing the inequality y<z in 0<x<y<z<N.) Assume, then, that C 
has at most hnitely many tangent lines which have contact of order > 2 with C, 
and for any J > 0 let Cg be the curve consisting of points of C at distance > J 
from each of those higher-order tangent lines. For each point P on Cg we obtain 
a lattice Lg{P) C R® whose nonzero short vectors correspond to points near P 
in R^(Q), of height ^ lying at distance ^ S from Cg. This gives a map 

Ag from Cg to the moduli space of lattices in R®. We would thus like to ask: as 
d — 7- 0, does the image of Ag become uniformly distributed in this moduli space? 

There are several problems with this formulation of our question. A minor 
one is that we have not dehned Ag precisely enough for the question to make 
sense, because we have left some O-constants unspecihed. This did not matter 
for qualitative properties such as whether the lattice has 0(1) short vectors, but 
makes it easy to frustrate uniform distribution by simply choosing Ag to avoid 
a small region in the moduli space. This problem is easy enough to hx for any 
given C] for instance, if C is given hy x [x \ y{x) : 1) for some differentiable 
function y : [0, 1] — [—1, 1] with bounded second derivatives, we may take for 
Ag{x) the integer span of the columns of 

/ 0 0 J 

10 -X 

\-'l//5 1/5 {xy'-yJ/S 




( 1 ) 
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But this brings us to a more serious difficulty. The question of whether As{Cs) is 
asymptotically uniformly distributed as J — 7 - 0 is likely to be a very hard problem 
in analytic number theory. For our purposes we are only concerned with how 
often and how close does As{P) come near the cusp of the moduli space. For 
instance, we see in the hnal section that if (7 is a conic then As{Cs) is restricted 
to a surface in the moduli space of lattices in R®, but within that surface it still 
approaches the cusp rarely enough that the average number of short vectors in 
a lattice in As{C) is still ^ log(l/J). In general, then, what we would like is 
the following result: as J 0 , the average number of vectors of norm < 1 of a 
lattice in As{Cs) is ^ log‘^*-^^(l/J). 

This still looks like a very difficult problem. While it remains open, we pro- 
pose a contingency plan in case the lattices Lg (P) have many more short vectors 
than expected. If all the short vectors are multiples of a single vector of small 
norm, there is no difficulty, because all these multiples yield the same point 
in R^. But there could be two independent short vectors, whose linear combina- 
tions yield a line in containing many points of small height near C. We claim 
that this is in fact the only way that a lattice of covolume ^ 1 could have more 
than 0(1) short vectors. This claim is easy enough to check using the descrip- 
tion of Minkowski-reduced lattices in R®, but we shall later need a generalization 
to lattices in higher dimension. We thus state and prove the generalization as 
follows: 

Lemma 1. For each positive integer n and positive real t there exists an effective 
constant Mn {t) such that the following bound holds: for any lattice A C R" whose 
dual lattice A* has no nonzero vector of length < r, and for any R > 0, there 
are at most Mn{rR) vectors of length < R in A. 

Here |T| is the covolume Vol(R"/T). The lemma can be obtained as a con- 
sequence of the theory of lattice reduction, but it is not easy to extract M„(f) 
explicitly this way. We thus give the following alternative proof in the spirit 
of [Cl] from which explicit (albeit far from optimal) bounds M„(f) may be eas- 
ily computed if desired. 

Proof. Given n, choose a positive Schwartz function / : R” —7- R with the fol- 
lowing properties: / is radial, i.e. f{x) depends only on \x\] and the Fourier 
transform / : R” —7- R, dehned for y £ R” by 

:= f f{x)e^^^^^’^Ux, ( 2 ) 



satishes f{y) < 0 for all y such that \y\ > 1. For instance, we may take 

f{x)^{\x\^ + a)e-^^\^\" (3) 



0 < c < 



27T 

n 



1 



a — 



n 

27Tc’ 



where 



( 4 ) 
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because the Fourier transform of a function (3) is 




for any c > 0 and a £ R. By Poisson summation, 

iceA ' ' yeA* 



( 5 ) 

( 6 ) 



Under the hypothesis on r, the only positive term in the sum over y is /(O). The 
sum over x is bounded from below by the sum over x of length < R, which is at 
least the number of such vectors times min|^l<^ /(ra?). It follows that A has at 
most 



m 

r”|/l| min|j,| <rRf{^) 



Mn{rR) r-"|T|-^ 



( 7 ) 



vectors of length < R, as claimed. 



Corollary 1. For each positive integer n there exists an effective constant A„ 
such that if a lattice A C R" has more than AnRA /\A\ vectors of length < R for 
some R> 0 then all those vectors lie in a hyperplane, which can be computed in 
polynomial time. 



Proof. Except for the last phrase, this follows from the previous Lemma by 
taking r — 1/R and A„ = M„(l), since then A* must have a nonzero y of length 
at most r, and any vector of A of length < R must be orthogonal to y. To assure 
that y can be computed in polynomial time, we take r — c/R for a positive 
constant c small enough that if A* has a nonzero vector of length at most c/R 
then the LLL algorithm will hnd a (possibly different) nonzero vector of length 
at most 1/R. Our Corollary now holds with A„ = c~'^Mn{c). 

From the case n = 3 of this Corollary we deduce: 

Theorem 1. Let C be the image of a differentiable map <f> : [0, 1] -P- RR^ 
with bounded second derivatives. Then for each N > 1 and 5 > N~^ one can 
find 0{SN^) rational points and 0{N) rational line segments each of length 
0{1/N) in R^ which together include all rational points of height < N at dis- 
tance < 6 from C. These points and line segments can be computed in time 
AC S N . Outside of 0{SN^logN) space used only to record each point 
or segment as it is found, the computation requires space AC log'^^^^A^. All implied 
constants depend effectively on C . 



Note that here we do not exclude neighborhoods of high-order rational tan- 
gents to C] such tangents will contain some of the lines segments computed by 
the algorithm. 
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2.3 Variations and Generalizations 



The problem of finding rational points near plane curves is only the hrst non- 
trivial example of many analogous problems to which our method can apply. We 
briefly discuss some of these here. 

One easy variation is to change the norm: instead of approximating the curve 
in the real valuation, use a nonarchimedean one, or a combination of several. For 
instance, one can eihciently seek nontrivial triples of small integers the sum of 
whose cubes is divisible by a high power of 2 or of 10. Likewise one can replace 
TL by F,j[T'] or similar rings in function helds of positive genus. The lattice- 
reduction step should then be even easier than in the archimedean case, though 
in the function-held setting our approach faces strong competition from the 
method of undetermined coeihcients, and it is not clear which is superior. All 
these comments apply equally to the adaptation of our method to the problem of 
hnding small nonzero values oi\x^ — y^\, provided the characteristic is not 2 or 3. 
For the \x^ — y^ \ problem, the work estimates are again rigorous; otherwise, they 
are still heuristic, but their analysis may be more tractable in the function-held 
case. 

Higher dimensions present many new opportunities. The easiest generaliza- 
tion is to a hypersurface in Here we are seeking small values of a ho- 

mogeneous function of k variables evaluated at an integral point. This time we 
chop the hypersurface into chunks each of diameter and 

replace each chunk by a subset of a hyperplane which approximates it to within 
0(J). The points of height < N that are within 0((j) of this chunk then come 
from integral points in a parallelepiped in whose sides have lengths 0{N), 
[k — 2 times), and 0{NS). Again most of these this parallelepipeds 
have volume 1 provided J N~^, and we locate the integral points using 
lattice reduction in R^. So, as long as J we expect to hud on the order 

of SN^ points, using ^ log‘^''^^^A space and spending ^ log‘^''^^^A time per 
point. For a general hypersurface, this again improves on other approaches to 
the problem. But the improvement decreases with k: the direct approach takes 
time log'^^^^A, and we lower the exponent by a factor no better than 

{k — 2)/{k — 1), which approaches 1 as oo. Moreover, lattice reduction in R^ 
quickly becomes diihcult as k grows. Another consideration is that for special 
surfaces there are known, and simpler, algorithms that take time log*^^^^ A 
or less once > 3. For instance, for Fermat surfaces in R®, one readily adapts 
the method of [Bl] to hud all solutions of x^ + y^ — in posi- 

tive integers with t < z < N, in expected time log*^^^^ A, and with no need 
for lattice reduction in R'^ or other complicated ingredients. This computation 
does require space proportional to A log A, which however poses no diihculty 
for practical values of A. As in the previous paragraph, all that is described in 
the present paragraph can be done also for a nonarchimedean norm, with similar 
results except that lattice reduction over a function held is tractable even for 
large k. In either case rigorous estimates may become even less accessible as k 
grows. 
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We can generalize further to manifolds A4 C of codimension c > 1. Here 
we expect to find on the order of rational points of height < at distance 
0(S) from M. We chop M into patches of diameter 

each of which yields a parallelepiped in with dimensions of order N (once), 
NS id times), and NS^!"^ (the remaining k —1 — c dimensions). We thus expect 
to efficiently find all ~ 5‘^N^ points as long as J ^- 2 fe/(fe+c-i)_ ^ further 
possibility emerges if JVL has bounded derivatives past the second derivatives and 
has small enough dimension compared with k: we can then make further headway 
when J falls below that threshold. Usually we are only interested in points much 
closer than pr-2fe/(fe+c-i). gg ggg only the structure we gain 

nothing by making J even smaller, so we may as well find all the points at 
distance 0 (A^“ 2 fe/(fe+c-i)^ g^^i^ locate the best approximations in the resulting 
list. However, if A4 is and its dimension d — fe— 1 — cisso small that 
k > , then a patch of diameter e is contained in a box with d sides of length 

^ e, a further (d^ + d)/2 sides of length ^ e^, and the remaining k — sides 
of length ^ e®. This means that we can makes our parallelepipeds thinner in 
some directions, and thus use wider patches of Ad, covering the entire manifold 
with fewer of them. This lets us locate the points of height < N closest to Ad 
in time significantly less than it would take to record all the points at distance 
^ pr-2fe/(fe+c-i)^ though not so efficiently that we only spend ^ log‘^''*-^^A 
time per point. More generally if Ad is a (J® manifold we can exploit bounds on 
the t-th derivatives once k > 

If the ambient projective space is not of high enough dimension, we can still 
make some use of approximations to Ad of degree t > 1 by using the t-th Veronese 
embedding Vj of into projective space of dimension — 1. [The t-th 

Veronese embedding takes the point with projective coordinates [Xi : • • • : Xk) 
to the point whose projective coordinates are all monomials of degree i 

in the Xj. Thus Vj raises all heights to the power i, and transforms intersections 
with hypersurfaces of degree d in into hyperplane sections in a projective 
space of much higher dimension. For more on Veronese embeddings, see for 
instance [FH], where they arise several times.] The idea is to surround each 
patch of Vj(Ad) by a box containing all points in Vj(IP^~^) at distance 0((I) from 
Vj(Ad). The resulting asymptotic improvement may be only barely worth it in 
practice, though. Consider the simplest case of a curve C £ R^, embedded in 
R® by V 2 • Assume for simplicity that the parametrization <j> of C has | </>" | bounded 
away from zero. Then, for J such that e^ <V. S <V. e^, the radius- J neighborhood 
in R^ of an interval of length e on C maps into a box in R® whose sides are of 
order e,e^,S,e^,e^. [To see this, choose coordinates [Xq : X\ : X 2 ) on R^ for 
which (f> is of the form [1 \ t \ + 0{t^)) for f in a neighborhood of 0, and 

note that U> takes (Aq : Xi : X 2 ) to (X^ : XgXi : X 0 X 2 : X^ : X 1 X 2 : A|).j 
Thus the points of height at most N in that neighborhood map to lattice points 
in a 6-dimensional parallelepiped of volume ^ (Here N occurs to the 

power 12 rather than 6 because V 2 squares the height of each rational point.) 
Thus if we take e — we expect to find all points at distance ^ S 

from C, of which there should be about SN^, in time N(SN^)^^^‘^ log‘^*-^^V. The 
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condition J e® yields S ^ so we save a factor of at most We 

pay not only by missing the points at distance between A^“36/i3 j \^-2 

usually do not interest us anyway) but also by reducing lattices of rank 6 rather 
than 3. This takes more time per lattice, and probably yields parallelepipeds 
whose average bounding box is larger. Each of these elfects amounts to only a 
constant factor, but these factors may be considerable, and it will be interesting 
to see how large N must be for this use of V 2 to be practical. 



2.4 Rational Points on Varieties 

In the last paragraph we exploited the fact that points near A4 map under Vj to 
points that are not only near Vj(Ad) but exactly on We can go much 

further when we search for points exactly on A4 . Again we consider the simplest 
case of a curve. We begin with a curve in one projective space: 

Theorem 2. Let C be an algebmie eurve in M -dimensional projeetive spaee, 
defined over Q and not eontained in any hyperplane. Then for any N >1 the ra- 
tional points ofC of height at most N ean be listed in time <^c log*^"^^^ A. 

The implied eonstants depend effeetively on d and C. 

Remarks, As seen above for M — 2, this result applies more generally to a 
curve in whose intersection with any hyperplane can be computed in poly- 
nomial time. The exponent 2/M is best possible: a rational normal curve of 
degree M (a.k.a. the image of P^ under Vm) has on the order of rational 

points of height at most N, and it takes time A^'^^^log A just to write them 
down. The constant implied in Od{f) and/or ^c, while effective, may be un- 
pleasant in practice for large M, since lattice reduction in dimension M -|- 1 is 
involved. 

Proof A segment of C of length ^ is contained in a box whose i-th side is 

^ _ \^2,. . M). The rational points of height at most A in this box 

come from points of contained in a box B whose i-th side is ^ 

{i — 0, 1, 2, . . . , M) and thus has volume 0(1). It takes time log‘^"*-^^A 
to apply lattice reduction and, by Corollary 1, either list fl i? or hnd a 

hyperplane containing fl B. In the former case, we test whether each of 

the resulting Ocf(l) points lies in C . In the latter case, we map this hyperplane 
to P^ and intersect it with C , Ending at most deg(O) = Oc>(l) rational points. 
Thus in either case we hnd all rational points of height < A on our segment in 
time <^c log‘^"^^^A. Since it takes only segments to cover C , we are 

done. 

It might seem that this algorithm is superhuous: if C has genus 0 then its 
small rational points may be found directly from a rational parametrization, 
without any lattice reduction; and if C has positive genus then we can hnd all 
its points of height < A in time ^ log'^^^^A once we have generators of the 
Mordell-Weil group of the Jacobian of C . But the difficulty is that we must 
hrst hnd these generators, and this requires locating rational points on a curve 
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or a higher-dimensional variety. For instance, to hnd the Mordell-Weil group of 
an elliptic curve E we usually apply a few descents and then search for points 
on certain principal homogeneous spaces for E, each of which is a curve C of 
genus 1, usually (in the case of a complete 2-descent) of the form — P{x) for 
some irreducible quartic P £ Z,[X]. One then searches for a? £ Q of height up 
to H for which P{x) £ There are on the order of candidates for x; one 
can set up a sieve to elhciently try them all, but this still takes time 
(and signihcant space) . Instead we can embed C in P® as the intersection of two 
quadrics (by writing P{x) as a homogeneous quadric in l,x,x^), and use the 
algorithm of Thm.2 with N — to hnd all rational solutions of — P(x) 
with X of height < id in time H . For certain E one can use Heegner 

points to locate a rational point on C to within J (see [E3]); if J ^ id~^, this 
is suflicient to identify x using continued fractions, a.k.a. lattice reduction in 
dimension 2. Using the new algorithm, we see that S suflices if we use 

lattice reduction in dimension 4. This saves a constant factor in the computation 
of X, since fewer digits and terms are needed in the hoating-point computation 
of Heegner points. When C has genus > 1, there are only hnitely many rational 
points by Faltings’ theorem, but they still may be of signihcant number and/or 
height. For instance, in [KK,S] one hnds curves C y^ — P{x) of genus 2 which 
have hundreds of rational points. In both cases, all points with x of height < 10® 
were found using the dd^ log®^^^^dd sieve method, a substantial computation. At 
least in the case considered in [S], where the Jacobian of C is absolutely simple 
with large Mordell-Weil rank, it would probably be even more onerous to hnd 
all these points by hrst determining the Mordell-Weil group. But the embedding 
{1 : X : x"^ : x^ : y) oi C into P'^ yields an improvement from dd^ to with 

5-dimensional lattice reduction. 

We can do even better by mapping the same curve to larger projective spaces. 
Fix an algebraic curve C of genus g dehned over Q, and a divisor dJ on (7 of 
degree d > 0. For n sulhciently large, the sections of nD embed C into 
This embedding sends any rational point on C of height (exponential, as usual 
here) < dd relative to dJ to a point on of height ^ dd”. By Thm.2 again, 

we can hnd all such points in time ^ ]^^n/{nd-g) Letting n — oo, we 

conclude: 

Theorem 3. Eix an algebraic curve C/Q and a divisor D on C of degree d > 0. 
Eor each e > 0 there exists an effectively computable constant such that for 
any dd > 1 one can find all points of C whose height relative to D is at most H 
in time Aedd(^/‘^)+^. 

For instance, all rational points on = P{x) with x of height at most dd 
can be computed in time ACe dd^+^. 

What of varieties A4 of dimension Zl > 1 in P^? A chunk of radius S 
then yields the intersection of with a box with sides as follows: one of 

length 0{N), A sides of length 0{NS), sides of length 0{NS^}, ..., 

sides of length 0{NSfi, ... until exceeds 

M . As usual we choose d so that the product of these sides is 1, and apply lattice 
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reduction to each of 0(5~^) chunks. The dilhculty here is that if the lattice is 
nearly degenerate, the hyperplane found in Corollary 1 meets JVL not in a hnite 
number of points but in a subvariety of positive dimension Zl — 1. This suggests 
an induction on Zl, since we can apply our method to that hyperplane section 
of jVL. But already for A — 2 such an argument requires a version of Thm.2 
with more uniformity in the implied constants than we know how to obtain. 
However, as with our hrst nontrivial case of curves in P^, we do not expect such 
degenerate lattices to arise in practice often enough to raise the computational 
cost above log'^^^^A^), except for a hnite number of proper subvarieties 

of Ad. If we assume this, we can again obtain better estimates by embedding 
Ad in larger projective spaces. Fix an ample divisor Id on Ad, and ask for all 
rational points whose height relative to Ad is at most H. Using the sections of 
nD to embed Ad in projective spaces, and letting n — oo, we hnd the following 
heuristic generalization of Thm.3: for each e > 0, there exists a proper subvariety 
Ado(e) of Ad such that all points of Ad — Ado(e) of height at most H relative 
to D can be found in time 



(^^(^((zl+l)/|Z>|)+.), 

where |dd| is the Zl-th root of the intersection number . One might even hope 
that Ado(e) can be taken independent of e. For instance, if Ad is a surface of 
degree d in P® then we expect that, for some union Ado of curves on Ad, we 
can hnd all rational points of height < A on Ad — Ado in time j\r( 3 /p 4 )+e_ 
We must admit that this is unlikely to yield a practical improvement over the 
A^log'^^^^A method we already knew: the hrst Vj that reduces the exponent 
of N below 2 is V 3 , and then (assuming d > 4) the exponent drops only to 24/13 
— but instead of reducing 4-dimensional lattices we are then faced with lattice 
reduction in dimension 20. It will probably be a long time before N can feasibly 
be taken large enough that this extra effort is worth the factor gained. 

Returning to plane curves, we can use this idea to prove an even stronger 
bound on rational points on a plane curve C that is analytic but not algebraic. 
This is because the homogeneous monomials of degree i in the coordinates of C 
are linearly independent for each i, so Vj(C') spans a projective space whose 
dimension grows quadratically in i (whereas for an algebraic curve the growth 
is always linear). This leads us to the following result: 

Theorem 4. Let C be a transeendental analytie are in P^, i.e. C — {f{x) : a < 
X < b} where f is an analytie map from a neighborhood of [a, b] to P^ whose 
image is eontained in no algebraic: eurve. Then for eaeh e > 0 there exists a 
eonstant sueh that for every H < 1 there are fewer than A^LT' points of 
height <H in C C\ P^(Q). 

Proof For each positive integer i consider Vj(C') C p(*A 3 d/ 2 _ Since C is tran- 
scendental, Vi{C) is an analytic arc Vjo/ contained in no hyperplane of P(® +3*)/2_ 
Now apply the argument for Thm.2 with N — H\ As noted in the remarks fol- 
lowing the statement of that theorem, the curve need not be algebraic as long 
as it is and its intersection with any hyperplane is of bounded size. (Here 
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we need not compute this intersection numerically, since we are only bound- 
ing the number of rational points of small height on C, not computing them 
efl&ciently.) The dilferentiability is clear since Vi{C) is analytic, and the bound- 
edness is proved in the next lemma. We conclude that the number of points of 
height < id on (7 is <^i id4/(*+3)_ i can be taken arbitrarily large, our 

theorem follows. 

The existence of an upper bound on the size of the intersection of any hyper- 
plane with Vi{C) is a special case of the following lemma in complex analysis. 
Throughout the lemma and its proof we count zeros of an analytic function ac- 
cording to multiplicity, even though in the application to Thm.4 a multiple zero 
is no worse than a simple one. 

Lemma 2. Let E be an open subset of C and V a finite-dimensional veetor 
spaee of analytie funetions: E ^ C. Then for any eompaet subset K C E there 
exists an integer n sueh that any nonzero f (E V has at most n zeros in K . 

Proof Fix K. We shall say that a compact K' Q E is “good” if its boundary 
d{K') is rectihable and its interior K' — d{K') contains K. Choose a good Ad, 
and dehne a norm on V hy ||/|| = sup^g^/ |/(.^)|. Let Vi be the unit ball {/ £ 
V : 1 1 /I I = 1}. It is sufl&cient to prove the lemma for / G Vi. 

For each / G Vi choose a good Kf C K' such that / does not vanish on 
d{Kf). Let ry — inCg^^jf^) l/(-^)l! ^ind let nf he the number of zeros of / in Kf . 
By Rouche’s theorem, if ^ G R with ||/ — (/|| < ry then g has at most nf zeros 
in Kf, and a fortiori in K. Now Vi is compact and is covered by the open balls 
Bf of radius ry about / G Vi. Thus there is a hnite subcover {Bffif^^. Then 
n maxj- nf^ is an upper bound for the number of zeros in K of any / G Vi, 
and thus of any nonzero A G R. 

To recover our result on hyperplane sections of R(C'), take K — [a, 6], let E 
be a neighborhood of K on which / is analytic, and choose any analytic functions 
fo,fi,f 2 on E such that / = (/q : /i : / 2 ) on E. Then take for R the space 
of homogeneous polynomials of degree i in fo,fi,f' 2 - If we understand / well 
enough to obtain for each i an effective bound n in Lemma 2 then the constants 
Ae in Thm.4 are effective too. 

With a little additional work Q can be replaced by an arbitrary number 
held E embedded in C, and C by f{K), where K C C is any compact subset 
and / is again an analytic map from a neighborhood of K to whose image is 
contained in no algebraic curve. 

A separate approach to bounding the number of rational points on curves 
was initiated in [BP] and pursued further in [P] and [HB2j. For example, Heath- 
Brown obtains in [HB2] bounds on the number of rational points on an algebraic 
plane curve that coincide with the time estimates in our Theorems 2 and 3. 
Moreover, our Thm.4 is contained in [P, Thm.8], which asserts that for a num- 
ber held E with [A : Q] = n the number of A-rational points of height < A 
on a transcendental analytic arc C is at most Ac\n,eH'' . Probably the methods 
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of [BP,P] can also prove these results with arcs C replaced by compact tran- 
scendental curves f{K), and our bounds can also be made uniform in F given 
[F : Q], There is clearly some overlap between the two approaches; for instance 
the Corollary preceding [P, Thm.8] is the same as our Lemma 2, but proved 
using the determinants of [BP,P], What is not clear, but intriguing, is whether 
those determinantal methods and our lattice-reduction technique can ultimately 
be interpreted as facets of the same basic idea. 

All this also suggests the question of whether a transcendental arc can contain 
inhnitely many rational points, of whatever height. I thank Michel Waldschmidt 
for pointing out that this question was already asked, and later answered aihr- 
matively, by Weierstrass. See [M2, Chapter 3] for this and related results. 

3 The Algorithm in Practice 

In this section we report on the outcome of the application of our algorithm to 
various plane curves, and on some results suggested by our hndings. We suppress 
details of the explicit constants replacing each 0{- ■ and these details are 
of course crucial in practice, but are straightforward and not enlightening. In 
each case our curve has some rational points of inflection, and we make sure to 
truncate our curve enough to avoid the tangents at such points but not so much 
that we lose approximations near but not on those tangents. 

In general, for a plane curve given by a homogeneous equation P{X, Y,Z) — 0 
of degree n, we associate to a rational point {x : y : z) near but not on the curve 
the number 

nmax(|;r|, \y\, \z\)^-^/\P{x,y, z}\, (9) 

which measures how close the point {x : y : z) is to the curve relative to the 
point’s height. We insert the factor n so that we can reasonably compare ap- 
proximations for curves of diiferent degrees. For instance, for the Fermat curve 
one expects that a,s x,y vary, the integer z~^ x^ + y^ comes on average within 
^nz'^~^ of the nearest n-th power of an integer, and thus that the smallest value 
of \z^ — J/" — x^\ for e [A, 2A] is proportional to nz‘^~^. One could insert 
further factors to correct for the length and shape of our curve, but these factors 
are not signiflcant for most of the curves we study. 

We noted already that the heuristics leading to formulas such as (9) refer to 
“random” {x : y : z) near the curve, not for systematic families of approximations 
which may attain values of the ratio (9) larger or more often than expected. We 
again give an example for the Fermat curves, which were the subjects of most of 
our computations. One usually guesses that for each r there will be ^ rlogA 
triples {x, y, z) such that the ratio (9) exceeds r. However, in the identity 

{t + 1)" - (f - 1)" = 2nf"-^ -h (10) 

we can make 2nf"“^ an n-th power by setting t — 2nw"; this yields 
triples with (9) bounded away from zero. We note the special cases n = 2, 3 of 
this identity: for n — 2, the error vanishes, and we recover a familiar 
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parametrizatioii of Pythagorean triples; for n = 3, the error is constant, and 
we can scale the identity to obtain the known family of solutions [x, y, z) — 
(6f^, — 1, + 1) of — y^ — — 2. Returning to general n: in our searches 

we set the threshold on z^~^ /\z^ — y^ — x^ \ low enough to hnd all the examples 
coming from (10), as a check on the computation; but we chose a higher threshold 
for the tabulation of results so that our list is not dominated by this polynomial 
family. 



3.1 Fermat Curves of Degree > 3 



We implemented our algorithm to hnd small values oi\z^ — y^ — x^\ with 0 < < 

y < z , A < n < 20, and £ [10®, 10®]. Since the threshold for “smah” depends on 
the size of z, we wrote [10®, 10®] as the union of 10 intervals [A^/2, A^] and treated 
each separately. We also used a direct search for < 5000, using the overlap 
region [1000, 5000] as a check on the computation. We did not attempt to hue- 
tune the algorithm for elhciency, since we carried it out more as a demonstration 
project than a major computational undertaking. Thus we programmed the 
search in gp, using the built-in arithmetic and LLL lattice reduction. We estimate 
that transcribing the program to C, and replacing LLL by Minkowski reduction 
in R®, would speed the computation by roughly an order of magnitude; of course 
a machine faster than a Sun Sparcstation Ultra 1 would help too. With a C 
program and a more powerful machine, it should be feasible to search the range 
n £ [4, 20], < 10® in time on the order of a month. 

The behavior of the run times and the counts of solutions with \z^ — j/” —x^ \ ^ 
z^~"^ seem broadly consistent with our heuristics, though we have not attempted 
a detailed statistical analysis. We tabulate the most striking examples, those with 

r := - y" - x^) (11) 



of absolute value at least 4: 

All decimal values of r are rounded to the nearest tenth. If for some integer 
A > 1 we have r > 4A® then [\x, Xy, \z) will also appear in the table provided 
\z < 10®; this happens for A = 2 at n = 4, 5, 7, 10, and for A = 3 at n = 5, 10. The 
hrst examples for n = 10 and particularly n = 5 (where 13®-|-16® = 17®-|-12) are 
small and striking enough that one feels they must have been observed already, 
but I do not know a reference. On the other hand, the hrst two examples for 
n = 12 have been published, and in a most unlikely place: each appeared in a 
different episode of the popular animated cartoon The Simpsons. Perhaps the 
third example for n = 12, or an example with n = 7orn = 15, could be used if 
the cartoon repeats this theme once more; the relative error \z^^ — y^ — x‘^\/ z'^ in 
each case is between 1 and 2 parts in 10^®, as compared to 3 • 10“^® and 2 • 10“^^ 
for the two four-digit examples. . . 
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n 


X 


V 


Z 


r 


n 


X 


V 


z 


r 


4 


167 


192 


215 


-4.5 


8 


209959 


629874 


629886 


-11.6 


4 


8191 


16253 


16509 


12.9 


8 


209945 


629826 


629838 


11.6 


4 


24576 


48767 


49535 


-64.5 


9 


6817 


10727 


10747 


5.3 


4 


49152 


97534 


99070 


-8.1 


9 


21860 


25208 


25903 


24.7 


4 


34231 


157972 


158059 


5.2 


10 


280 


305 


316 


137.1 


4 


76215 


311390 


311669 


-14.8 


10 


560 


610 


632 


17.1 


5 


13 


16 


17 


-120.4 


10 


840 


915 


948 


5.1 


5 


26 


32 


34 


-15.1 


10 


7533 


8834 


8999 


4.4 


5 


39 


48 


51 


-4.5 


12 


1782 


1841 


1922 


6.1 


5 


42 


71 


72 


-8.8 


12 


3987 


4365 


4472 


-7.1 


5 


262 


328 


347 


-6.2 


12 


781769 


852723 


874456 


10.3 


5 


1125 


2335 


2347 


-5.0 


13 


666 


806 


811 


8.3 


5 


5088 


16155 


16165 


4.1 


13 


5579 


8235 


8239 


4.1 


5 


190512 


292329 


298900 


5.5 


15 


434437 


588129 


588544 


42.9 


6 


1236 


3587 


3588 


12.5 


16 


492151 


741267 


741333 


4.6 


6 


6107 


8919 


9066 


-9.9 


19 


79 


85 


86 


-4.7 


7 


386692 


411413 


441849 


78.4 


19 


491 


565 


567 


4.9 


7 


773384 


822826 


883698 


9.8 


19 


43329 


51144 


51257 


5.8 












20 


4110 


4693 


4709 


4.3 



Frivolity aside, one is struck by the pair of examples for n — S. The values of 
r are far from the largest in the table, but they are almost equal and opposite, 
and involve nearly equal triples {x, y, z) for which z — y has the same small value 
of 12. This suggests that we are dealing with a polynomial family {x{t), yit), z{t)) 
specialized at f = ±fo • Indeed we quickly hud that these are the cases t — ±3 of 

(32f® + mf + (32f® + 7)® = (32f® + lOtf + 21 • 2^^^° + (12) 



with r = f^/21 + Thus arbitrarily large values of r occur, and indeed 

z^ — y^ — can be as small as 0{z^'^l'^) rather than the expected 0{z^^). Trying 
to generalize the identity (12) further, we soon hud that there are similar families 
for any exponent n such that 3n(n — 2) is a square: 

Theorem 5. Let n > 1 be a positive integer. Then there exist polynomials 
x{t),y{t),z{t) e Z[f] of the form 

x(t) = Cf" + D, y(t) = + Bt, z(t) = + B't (13) 

with A D, B' B sueh that z^ — y^ — x^ is a polynomial of degree at most 
n{n — 3), if and only ifin{n — 2) is a square. In that ease, there exist infinitely 
many integer triples [x, y, z) with 0 < x < y < z sueh that z’^ — y’^ — x’^ AC 
^{A — 3n }/ (n + 1) 

Proof. Let b,¥ be the distinct rational numbers B / A, B' / A. Expand z^ — j/” at 
inhnity: 



f '2 Ti 1 2 

- y" = nA^{b' - b) U" + ^— (&' + bK 
+ (n - l)^(n - 2) (^,2 ^ 



( 14 ) 
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For this to be of the form (Cf" + £>)" + ®") we must have 

(n-1) (^^(6^ + 6)^ {b'^ + b'b + b^}. (15) 

The discriminant of this quadratic equation in ¥ /b is 3n(n — 2) times a square; 
thus (15) has nonzero rational solutions if and only if 3n(n — 2) is a square. 
Explicitly we hnd that b, b' are proportional to \/ {r¥ — 2n)/3 ± 1. 

Conversely, suppose r¥ — 2n — im? for some integer m. Let 

z — + c(m + l)t) , j/ = A(f""*“^ + c(m — l)f). (16) 

Then 

( f) 1 \ ^ 2 

f" + — — cmj +0(f"-3"). (17) 

To make this (Cf" + £>)" + 0(f”^“®”) with C,D EA, we now need only choose 
nonzero c £ Z so that 2cn is an n-th power (e.g. take c — (2n)”“^), and 
then choose A so that n|Acm. Specializing t to sufl&ciently large integers in 
the resulting {x{t),y{t),z{t)) yields inhnitely many integer triples {x,y,z) with 
b) < X < y < z such that z^ — y^ — x^ ^ ^(n^-3n)/(n+i)^ claimed. □ 

The smallest n > 3 such that r¥ — 2n — im? is n = 8. There are inhnitely 
many further examples, starting with 27, 98, 363, . . . , and parametrized by a 
Fermat-Pell equation. Dropping the constraint n > 3 yields the further cases 
n — 2 and n = 3. For n — 2 we again obtain a Pythagorean parametrization, 
this time with x,y,z multiplied by t; for n = 3 we hnd 

(9f^ + If + - (9f^ + itf = 1, (18) 

one of inhnitely many polynomial solutions oi x^ + y^ — z^ — 1. 



3.2 The Fermat Cubic 

Our algorithm applies to the Fermat cubic as it does to the Fermat curves of 
higher degree, but we treat it separately both because the heuristic analysis 
is subtler and because the problem of hnding small values oi \z^ — y^ — x^\ 
has already attracted some attention. We noted that in general we expect the 
smallest values of \z^ — y^—x^ \ to be comparable with For n = 3, we have 

z^^~^ — 1, and of course (given this case of Fermat’s Last Theorem) \z^ — y^ —x^\ 
can be no smaller than 1 for nonzero integers x, y, z. Moreover, z^—y^—x^ cannot 
be an arbitrary rational multiple of only the discrete values ±1, ±2, . . . may 
arise. Thus, instead of a Diophantine inequality z^ — y^ — x^ ^ z~^~^, we have 
a family of Diophantine equations z^ — if — x^ — d (d (E A), and new tools can 
bear on solving them or, failing that, describing their distribution of solutions. 
These equations have been investigated by various means since the beginning of 
the computer age; see [G] for references to work up to about 1980 (some of which 
dates back to the 1950’s), and [B2,CV,HBLR,KTS,PV] for more recent results. 
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As we shall see, the problem has been approached in several ways, some of which 
already improve on direct exhaustion over some values of {x,y). Still, our 
new linear approximation method is better yet, both in heuristic theory — even 
though by factors smaller than our accustomed A/log'^^^^A — and in practice, 
as evidenced by the computation of many new solutions. Our discussion here 
applies with almost no change to other “diagonal” cubics, such as + + 2z^ 

which was also singled out in [G, Prob. D5]; but we have not yet implemented a 
search for small values of + j/® + 2z^j beyond what has already been reported 
in the literature. 

For each nonzero d, the expected distribution of solutions of 

z^ -y^ - x^ - d (19) 

involves not only considerations of size — i.e. of local behavior at the archimedean 
place of (Q — but also on the behavior of z^ — y^ — x^ at hnite primes p: each p 
contributes a local factor fp (d) that is the ratio of the p-adic measure of the Ap- 
points of (19) to the average of that measure as d ranges over Ap. For instance, if 
any of those factors fp{d) vanishes, there can be no solutions at all. It is not hard 
to see that the only such local constraint is d ^ ±4 mod 9. For such d, the re- 
sulting product over p was investigated by Heath-Brown [HBl]. He showed that 
the product does not converge absolutely, but can nevertheless be analyzed and 
approximated numerically by comparing fp (d) with the factor at p of the Euler 
product for (C(Q)(.J4f)(®)/C(®)) at s = 1, which differs from /p(d) by a factor of at 
most 1 -|- 0{p~^/'^). The product ](([p /p(<^) is then seen to diverge to -|-oo if d is a 
cube and to converge to a positive limit when d is neither a cube nor congruent 
to ±4 mod 9. Heath-Brown thus conjectured in [HBl] that all nonzero integers 
d ^ ±4 mod 9 occur as z^ — y^ — x^ inhnitely often. So far this is only known 
when d is either a cube or twice a cube, thanks to polynomial parametrizations, 
which the above heuristics do not try to account for. We have already exhibited 
polynomial solutions for d = 1,2. For many d ^ 4 mod 9 which are neither cubes 
nor twice cubes, not a single solution is known for z^ — y^ — x^ — d. Heath-Brown 
observes [HBl] that this is not surprising, because for many of these d the ex- 
pected number of solutions with £ [N, 10® A] is positive but smaller than 1. 
Guy [G] lists the cases with d < 10® which were open as of 1980, and while the 
list is now shorter the question of which integers are the sums of three cubes 
is not yet settled even in that range. For instance, the case d = 30 was open 
until 1999, and had been the smallest open case for several decades. 

We have noted already that a direct search hnds all small \z^ — x^ — y^\ with 
< A in time A®log®^^^^A. There have been several improvements on this, all 
obtained by rewriting the equation (19) as 

x^ + d^z^ -y^ ^{z-y){z‘^ + yz + y'^). (20) 

Once x^ + d is factored, which takes heuristic time f all solutions of (20) 
can be found by trying each factor of x^ + d for z — y. Given the value of d, this 
takes time only In addition to dealing with only one d at a time, this 
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method has the disadvantage that the time required to factor x^+d, though 
subexponential, is still considerable. The advantage of this method is that it hnds 
all solutions with x < N, while y, z may be considerably larger, of order up to 
/f3/2_ Many of the new solutions found in [KTS] are of this type, with y, z large 
but z — y very small. Heath-Brown observed that, again given d, the factorization 
of x^ + d can be simplihed by a precomputation in Z[-^], though the complexity 
of the precomputation depends unpredictably on d via the arithmetic of the 
number held Q{Vd); this approach was implemented in [HBLR]. Note that in 
ehect these methods hud rational points near the Fermat cubic that are close 
to the tangents to the curve at its inhection points — the same tangents that 
demand special care in our algorithm. A further variation which we suggested 
in 1996 is to use the factorization 

z^ - d — x^ + y^ — {x + y){x^ + xy + y^) (21) 

as follows: hx x+y, solve for mod x+y, and try each of the resulting values of 
Here we only hud solutions with z, not x, bounded by N, but the advantage is 
that factoring costs are greatly diminished. To hud all cube roots of d mod x + y 
requires factoring x + y, a, number of size N rather than N^; and with enough 
space to set up a sieve the factorization can be avoided entirely. In 1999, Eric 
Pine, Kim Yarbrough, Wayne Tarrant and Michael Beck, all graduate students 
at the University of Georgia, took up this suggestion, choosing d — 30, and found 
the hrst solution: 

30 = 2220422932^ - 283059965® - 2218888517® (22) 

We announced our new algorithm in the same 1996 posting to the NMBRTHRY 
mailing list, together with results of a search for solutions with < 10^ and 
|d| < 10®. We did our search in gp, making our computation easy to program 
(since gp already provides multiprecision arithmetic and lattice reduction) but 
far from optimally elhcient. In 1999, unaware of the work of the Georgia group, 
we asked Dan J. Bernstein for an elhcient implementation. He soon wrote a C 
program that found all solutions with z < i ■ 10® and |d| < 10'^, including (22) 
and many others. Several values of d had not been previously represented as 
the sum of three cubes. Detailed results and analysis will appear elsewhere. As 
usual, since we are interested in small d, not all d <+ N, the improvement by a 
factor should apply here as well to hud all cases of \z^ — y^ — x^\ z^^^^ 

with z < N, but we have not attempted to implement such a computation. 

3.3 Miscellaneous Examples 

Trinomial Units. One sometimes sees in Olympiad-style mathematics contests 
the question “Is z^^^ greater or smaller than x^^^ + y^l^T’’ for some specihc 
positive integers x,y,z. Of course this is a challenge only when the sign of the 
difference z^/^ — {x^/^ + y^^^) cannot be determined by inspection. In 

some cases the question be settled by applying classical inequalities; for instance 
if a > 6 > 0 then (a -|- 6)^/® -|- (a — 6)^/® < 2a^/® by convexity of the cube root. 
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The general solution is to compute the norm of u^j an algebraic number of 
degree 9 none of whose other conjugates is real unless x — y. We hud that W 3 
has the same sign as 



N(a?, y, z) [z — y — x)^ — 21xyz. (23) 

Moreover, given the size of x, y, z, the smaller N(a?, y, z) is, the nearer W 3 will be 
to 0. In particular, we would like to have N(a?, y, z) — ±1, which would make the 
algebraic integer W 3 a unit. Thus again we seek rational points close to a plane 
cubic curve, here N(a?, y, z) — 0. This time the curve is rational: by construction, 
it is parametrized hy [x \ y \ z) — : (1 —t)^ : 1). It is thus not smooth, but its 

only singularity is the isolated point [x \ y \ z) — [1 \ —1 \ — 1 ) (geometrically a 
node with complex conjugate tangents), which does not aifect our algorithm. The 
three rational points of inflection at xyz — z — y — x — {i do aifect our algorithm, 
but fortunately we are not interested in the points on their tangent lines, since 
those are the points with xyz — 0. We thus restrict our attention to the portion 
of the curve with xjz, yjz > 1/N , i.e. with t ^ and 1 — f in 

the rational parametrization. This takes us far enough from the inflection points 
that they cause us no difficulty. 

The situation is now much the same as for z^ — y^ — x^ — d. We expect the 
number of solutions of N(a:, j/, .?) = d of height up to N to be proportional to 
log N times a product of local factors gp [d ) . The only local factor that can vanish 
is g^{d), which is nonzero if and only if 9|d or d = ±1 mod 9. We henceforth 
assume that d is in one of these congruence classes. We can then check whether 
rip dp converges by comparing it with the L-series of the projective cubic surface 
N(a:,j/, .?) = dt^. This in turn depends on the Galois structure of the Neron- 
Severi group of the surface, which can be determined from the action of Galois 
on the lines on that cubic surface, as explained in [Wl]. We must be careful 
here because, unlike x^ + + z^ — dt^, the surfaces N(a:, j/, .?) — dt^ are not 

smooth: each has an A 2 singularity oX [x \ y \ z \ t) — [1 \ —1 \ —1 \ d). Thus 
each has, not 27 lines as usual, but 15, of which 6 go through the singularity; 
see [BW]. Explicitly, these are the preimages under the projection io [x \ y \ z) 
of the three coordinate axes and the two tangents to the curve at (1 : — 1 : — 1 ). 
We conclude that, as with (19), Yipdpi'^} converges unless d is a cube. So we 
expect the number of unparametrized solutions of height < to grow as log A^, 
except when d is a cube, when it should grow faster, albeit still as a power of 
log A^ — perhaps log® N, by analogy with Manin’s conjecture for cubic surfaces. 

Unlike the case of (19) , we know of no solutions of N(a:, j/, .?) = ±1 in noncon- 
stant polynomials x,y,z ^ 2i[f], other than the trivial ones with xyz — 0. Never- 
theless we can And inflnitely many nontrivial integer solutions parametrized by 
Fermat-Pell equations, and thus show that the number of solutions of height < N 
is 5^ logN. There are several ways to do this. In 1982 we found a somewhat 
complicated route to such a parametrization, obtaining a family of solutions 
starting with N(16948, 31226, 186919) = —1. The details may be found in the 
pages of [CM]. Many years later, we observed that a simpler approach is to factor 
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N(a?, y, z) — ±1 as 

21xyz - {z - y- xfzs^l- {z - y- xz\:l)[{z - y-xf ±{z - y -x) + l\. (24) 

For each r £ Q* , we obtain a conic curve Cr by setting {z — y—xz\^l) — rx in (24). 
This can be viewed geometrically as follows: the aflhne surface N(a:,j/, .?) = ±1 
contains the line x — [z — y — x thus the intersection of the surface 

with any plane [z — y — x^s^X) — rx containing that line is the union of the line 
and some residual conic, which is our Cr- Likewise we could start from the line 

= [z — y — xzs^l) = 0 and intersect it with a variable plane [z — y — x^s^l) — rz. 
For many choices of r, one of these conics is a hyperbola with inhnitely many 
integral points parametrized by a Fermat-Pell equation. 

In retrospect this approach to N(a:,j/, .?) = ±1, in which we hber an aflhne 
surface by conics that may be regarded as principal homogeneous spaces for 
Fermat-Pell equations, seems a remarkable premonition of our later analysis [El] 
of the projective quartic surface via a hbration by genus - 1 

curves (principal homogeneous spaces for elliptic curves). In both cases the ap- 
proach hnds inhnitely many solutions but does not readily lend itself to eflh- 
ciently hnding all solutions of height < N. Again a later computation found 
that the solution that was discovered hrst, because it lies on the hrst hber that 
could contain a solution, is not the one of smallest height. We used our algo- 
rithm to hud all small values of N(a:,j/, .?) with 0 < x,y,z < 10®. We found 
that the smallest solution of N(a:,j/, .?) = ±1 is (14,84,313) of norm +1, fol- 
lowed by (6818, 4996, 46879), (20388, 4881, 86830), and (2742, 32540, 96843) each 
of norm -1, the known (16948,31226,186919), and (3408,182899,370338) of 
norm +1, with no further solutions up to 10®. We also found several primitive 
solutions of N(a:, y, z) — ±8 and a few sporadic examples with d small but not 
a cube, which could not have been obtained at all using the factorization trick; 
the smallest of these are 

N(204, 115327, 162434) = 17, N(650, 1425, 7899) = 26. (25) 

The N(a:,j/, .?) = 17 solution yields a disappointingly large value of because 
the conjugates z^!^ — y^!^ — are smaller than usual. An unexpected 

result — since the identity ( 10 ) cannot be used with exponents <1 — was 
a polynomial solution of N(a:,j/, .?) = 108, namely (4, j/(f), — j/(— 1 —t)) where 
y{t) — 4f® — 6 f -|- 3. We can write this symmetrically as ( 8 ,^(f), where 

git) — St^ — 12t — &t + 11, a cubic polynomial determined up to scaling by the 
condition that the Laurent expansion at inhnity of {y{t))^^^ have vanishing t~^ 
and t~^ terms. In this form, N(a:,j/, .?) is the larger constant 864 = 2^108, but 
with the bonus that a: is a cube so involves one fewer surd; for instance, taking 
t — 7 we hud that v^3279 is smaller than 2 + 7>\/Yf by less than 3.75-10“^. In this 
family, as with the hrst example in (25), is of order z~'^ , not z~^!^ , because 
two of the conjugates of are 0 ( 1 ). 

A similar investigation of W 4 := z^!^ — [x^/^ + y^^^) was not as productive, 
perhaps not surprisingly since there are no arithmetic reasons to expect many 
nonzero small examples. For the record, the smallest value found for 
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^ < 10® was 0.365+ for {x,y,z) — (241,691,6759), while the smallest |w 4 | in 
that range was (3.23-) • lO"^® for (37792,36109,591093). 

The 7T-th Fermat Curve. To illustrate our algorithm also for non-algebraic 
curves, we chose to apply it to the Fermat curve of exponent tt. Since tt exceeds 
3, but only slightly, we expected that \z'^ — y'^ — x'^ \ achieves a global minimum 
over all x,y,z with 0 < x < y < z but that the minimum might involve numbers 
of several digits. We were rewarded with the example 

2063’" + 8093"" - 8128"" = 0.019369- = 8128’"-7(184.75+), (26) 

which seems likely to be the minimum oi \z'^ — y'^ — x'^ \ over all positive integers 
X, y, z. At any rate, according to our computations it is the smallest with < 10®. 
The ratio 184.75+ is also the largest in that range, though there is also 

1198’" + 4628"" - 4649"" = -(0.04949+) = -4649’"“®/(66.794+). (27) 

It will probably be a long time before the question of the minimality of (26) is 
settled; a weaker but still intractable conjecture is that there are only hnitely 
many integer solutions oi \z'^ — y'^ — x'^\ < 1. 

The Klein Quartic. All our examples so far were Fermat curves, even 
though some had unusual exponents 1/3, 1/4, tt. Probably the best-known pro- 
jective plane curve that is not a Fermat curve is the Klein quartic K (A, +, Z) = 0, 
where 

K{X, Y, Z) := X^Y + Y^Z + (28) 

We used our algorithm to search for small values of K{x,y,z). By symmetry 
we may assume max(a?, y,z) — z. We are then seeking rational points near 
a segment of a plane curve with a single inflection point, at x — y — 0. 
The tangent x — t) at this point accounts for the obvious family [t),l,z) with 
K{x, y, z) — z. Our computation up to height 10® quickly revealed a less obvious 
family, with K{x,y,z) growing even more slowly than the 

height. As usual we also found sporadic examples, though here (as with several 
other cases we have already seen such as the Fermat quintic) the best ones are 
small enough that our algorithm was not needed to locate them: 

A(1421, -1057, 1501) = -49, 

A(7211, -8381, 11010) = -121, (29) 

A(-1550, 11817, 32615) = 245, 

with z/\K{x,y,z)\ — 30.6, 91.0, 133.1 respectively. The largest z/\K{x,y,z)\ 
found with £ [10®, 10®] off the singular cubic j/® + x"^z — 0 was 6.756+, from 
A(-7871, 175577, 829244) = 122741. 
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4 Hall’s Conjecture 

4.1 Review of Hall’s Conjecture 

By HalVs conjecture me mean the following assertion: if x, y are positive integers 
such that 

k (30) 

is nonzero (equivalently, such that {x,y) then 

\k\ x^l’^~‘^. (31) 

(While this accords with current usage, it is not exactly what Hall originally 
wrote: as F. Beukers points out. Hall [H] conjectured |fe| 5^ x^!"^ , a stronger 
statement which is probably false — the usual heuristic suggests that there are 
at least (J + o(l)) logdf cases of 0 < |fe| < 5\fx with x < X — but unlikely to be 
soon disproved. See also [BCHS] for the early history of this conjecture.) Among 
several equivalent forms of (31) we note the conjecture that the discriminant 
of an elliptic curve over Q in its standard minimal form has absolute value 
■ Known lower bounds on |fe| are much weaker than (31). By Siegel’s 
theorem on the hniteness of integer points on elliptic curves, each nonzero £ Z 
occurs hnitely many times as a:® — j/^, so |fe| — oo as a: — oo. Siegel’s proof is 
ineifective and thus says nothing about how fast |fe| must grow with x. Starting 
with Baker’s method, eifective bounds have become available, but they are still 
very weak. For instance, it is not yet possible to prove for any d > 0 that 
\k\ > x^. 

Hall’s conjecture is now recognized as an important special case of the Masser- 
Oesterle ABC conjecture [O] (see also [L]). Thus its analogue over function helds 
is known to be true by Mason’s theorem [M3]. In the special case of Hall’s con- 
jecture for polynomials x{t), yit), the fact that x^ — y^ is either zero or has degree 
> ^ deg (a:) was proved some twenty years earlier by Davenport [D2] in response 
to a question raised in [BCHSj. As in [E2] it follows that the conjecture cannot be 
disproved by a polynomial parametrization, and indeed in any polynomial fam- 
ily {x[t),y{t)\t £ Z) we must have k ^ x^ with 9 > 1/2. One does better with 
solutions parametrized by Fermat-Pell equations, i.e. x,y ^ Z[f, + bt + c] 
for some a,b,c ^ Z such that = at"^ + bt + c has inhnitely many solutions. The 
function held Q{t, \/at^ + bt + c) is then still rational, so the Davenport-Mason 
inequality again holds, but since now there are two places at inhnity one can 
have x^ — y"^ of degree exactly |deg(a:), and thus attain 9 — 1/2. The existence 
of a single such family (exhibited below) shows that the exponent in (31) cannot 
be raised above 1/2. The fact that one cannot reduce 9 below 1/2 in this way was 
again observed in [E2] in the more general context of the ABC conjecture. This 
fact lends some credence to that conjecture, and thus to its special case (31); 
this contrasts with the situation for — j/” —x^\, where there is no reason why 
some polynomial or Pell family might not do better than the expected 

by probabilistic heuristics, and indeed we found such families for some choices 
of n. 
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We next digress to say some more on polynomial and Fermat-Pell families 
that attain the Davenport-Mason bound, both because they are of independent 
interest and because families of both kinds appear in our numerical results. In 
either case — k is an identity in a genus-zero function held, namely 

Q(f) in the polynomial case and Q(f, + bt + c) in the Fermat-Pell case. Let 
X, y have degrees 2m, 3m respectively, and suppose k has the smallest degree 
possible, i.e. m -|- 1 in the polynomial case and m for Fermat-Pell. Then / := 
x^/y"^ is a rational function of degree 6m or 12m on ramihed only above 
0, 1, oo. The Riemann existence theorem provides inhnitely many such functions 
/ = x^ jy^ in C(f); this answers the hrst part of the question raised in [BCHS, 
p.68]. The second part concerns solutions over R, and can probably be settled 
by adding data on complex conjugation to the branched covering. But we are 
most interested in the third part of the question, in which / must have rational 
coelhcients. Given any one {x{t),y{t)), we may trivially obtain others of the 
form [x\]/) — {X^x{f),X^y{f)) where f — at + b in the polynomial case, and 
f e Q[t] with V + bt' + c/V + bt + c £ Q\t] in the Fermat-Pell case. If we 
regard such {x\ i/) and [x, y) as equivalent, only a handful of examples over Q 
are known, and there may well be no others. We next list representatives of the 
known examples. 

In the polynomial case, all known examples have m < 5. For m = 1, trans- 
lation and scaling brings any quadratic x{t) to the form -|- 2a, and then 
y — t^ -\- iat and k — ia?t^ + 8a®. Necessarily a _0, and all such examples 
are “twists” of each other, becoming isomorphic over Q if not over Q. Note that 
x^jy^ is a degree-3 function of with a triple zero. This function occurs for 
instance as the cover of the modular curve X(l) by Xq( 2). For m = 2 we again 
hnd that the solution is unique up to twist: x — t'^ + Aat, y — -|-6af®-|-6a®, and 

k — — 8a®f® — 36a'^. This time x^/y"^ is a degree-4 function of f®, whose ramihca- 
tion identihes it with the modular cover Xq( 3) — X(l). Birch found examples of 
{x, y, k) with m = 3, 5 and included them in a 29. ix. 1961 letter to Chowla; they 
are reported in [BCHS]: 



fset® -h 24t"‘ -h lOt^ -h 1, 216t® -h 216t^ -h 126^ -h 35t® -h 



21 . 



9 4 39 2 

-G -I — r -h 1 



and 



(32) 



/ 1 - P® 

fi(t®+6t® + 15t® + 12), ^— + 



+ 4p + 8t® 



-f 



5G -h 1 



3t® -h 141® -h 27 
108 



(33) 

These yield integer solutions if f is a multiple of 4 in (32) or congruent to 3 mod 6 
in (33) . As noted in [BCHS] , the second example provides inhnitely many integer 
solutions of I a:® — j/® I ^ moreover, for this choice of twist, the leading coef- 
hcient of k{t) is small enough that \x^ — y^\ is even a respectably small multiple 
of x^!"^ for the hrst few specializations of t. The maps / = x^ jy^ associated with 
Birch’s polynomials both have interesting Galois groups. For (32), / is a degree-9 
function of whose Galois group is PSL 2 (Fs) over C(f®) and Aut(PSL 2 (Fs)) 
over Q(t®); the Galois closure is the Fricke-Macbeath curve [F,M1]. For (33), 
/ is a degree-10 function of f® whose Galois group is PSL 2 (F 9 ). These groups 
and curves do not arise in connection with classical modular curves, but they 
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can be identified with certain Shimura modular curves, most naturally those 
associated with with the (2, 3, 7) and (2, 3, 8) arithmetic triangle groups (see for 
instance [T,E5]). Hall [H, p.l85] gives an example with m = 4: 

a? = 4(t® + + 2lf + + 109f2 + 74t + 28); (34) 

In August 1998 I announced a new example with m = 5 (its computation will 
be explained elsewhere): 

a: = t^“+2t®+33t®+12t^+378t®-336t®+2862t'‘-2652t®+14397t^-9922t+18553. (35) 

In both cases (as with all the other [x,y,k) examples), y is obtained by trun- 
cating the Laurent expansion at infinity of after the constant term. Neither 
(34) nor (35) yields an interesting Galois group: the Galois groups of x^ jy^ are 
Alt 24 and Sym 3 o respectively. While (35), like (33), must yield infinitely many 
integer solutions of \x^ — J/^| ^ x^!^\ the leading coelficient of k in (35) makes 
the implied constant much larger, and none of these solutions will appear in our 
list of small values of \x^ — y^\- The question, raised in [BCHS], whether there 
are any x,y,k ^ Q[f] of degrees 2m, 3m, m -|- 1 with m > 5, remains unsolved. 

For Fermat-Pell families, the list is even shorter: all known examples are 
equivalent, and come from the identity 

(f2 -h lOf -h 5)^ - (f2 -h 22f -h 125) (f2 -h 4f - if = 1728b (36) 

Here j/ is a multiple of ^/aI^~+Tf+~c , so / factors as a map of degree 6 composed 
with the double cover of Q{t) by Q{t, + bt + c). We noted in [E4, p.49] that 
the resulting degree-6 map / = x^/y"^ : is the cover Xq( 5) — X(l) 

of classical modular curves. Thus the elliptic curves of low discriminant coming 
from the identity (36) all admit a rational 5-isogeny. Each Fermat-Pell family 
obtained from (36) by specifying the class of + 22t + 125 mod yields 
k ~ Cx^!"^ for some nonzero C. The smallest such C is 5“®'^^54 = .96598 . . ., 
obtained by Danilov [Dl] by substituting 125(2f — 1) for t in (36) and dividing 
by 20^: 

(5'^f2-3000f-h719)^-(5^f2-114f-h26)(5®f2-5^123f -73781)2 ^ 27(2f-l). (37) 

The factor — 114f -|- 26 is a square for t — —5, and thus for infinitely many t. 
The first case t — —5 of this yields the elliptic curve of discriminant —11 labeled 
11-A2(C) in Cremona’s table [C2]; it is known that the isogeny class of this 
curve provides the examples with minimal conductor of a rational 5-isogeny, 
and indeed of an elliptic curve over Q. 



4.2 The New Algorithm 

To obtain numerical data with which to compare Hall’s conjecture, we want to 
find all small nonzero values oi\x^ — y^\ with x,y and a: < A. So that we can 
compare our algorithm with other approaches we briefiy review previous work 
on this problem. 
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The most direct approach is to simply compute for each x < X the integer y 
closest to Since x^^^ varies smoothly with x, this can be done quite eflh- 

ciently, but clearly must take at least time proportional to X. This is essentially 
what Hall did in [H], with X = 7 • 10®; some three decades later, faster comput- 
ers make larger X feasible, and indeed Frits Beukers reports in an Aug. 1998 
e-mail that he performed such a computation for X — 10^^. But this is probably 
close to the practical limit with today’s technology, and at any rate this direct 
approach is superseded by the X^^"^ log'^^^^A algorithm described below. 

A fundamentally different approach is taken in [GPZ]: for each nonzero k £ 
[—A', A'], investigate the arithmetic of the elliptic curve Eh : y"^ — x^ — k, and 
use effective bounds on integral points to hnd all integer solutions of x^ — y"^ — k. 
In [GPZ], Gebel, Pethb, and Zimmer did most of this work for K — 10®, except 
for a few values of k, for which they could not hnd a generator for Eh{Q); 
Wildanger later showed in his doctoral thesis [W2] that none of these Eh has 
an integral point, thus completing the computation of integer solutions of 0 < 
\x^ — y^\ < 10®. It is not clear even heuristically how this method compares 
with other approaches. It is the only approach used thus far that will provably 
hnd all solutions with |fe| < K . (The recent proof of the modularity conjecture 
means that Cremona’s algorithms [C2] yield another such approach, but to my 
knowledge it has not been used to solve x^ — y^ — k.) Assuming Hall’s conjecture, 
|fe| < K is equivalent to x A'^+^, but this begs the question of the constant 
implied in Neither do we know how to estimate the average work required 
to hnd all integer points on a curve Eh ■ It may be reasonable to guess that this 
average work is proportional to for some c > 0. (This estimate certainly 

holds for Cremona’s algorithms.) The total work would then be . Under 

Hall’s conjecture, this is equivalent to , so strictly worse (modulo an 

unknown implied constant) than our X^!"^ log‘®^^^^A algorithm, though perhaps 
better than a direct search, depending on whether c < 1. 

We noted already that the direct search can exploit the smoothness of the 
function x^!"^. We can try to take further advantage of this by mimicking our ap- 
proach to rational approximation of curves: surround the segment x < X of the 
semicubical parabola y — x^!"^ by a union of parallelograms each of area 0(1), 
and use lattice reduction to quickly hnd all integer points in each parallelogram. 
This does give an asymptotic improvement, though a small one: the parallelo- 
gram containing a point [x, x^!"^) has length so the computational cost 

is reduced by at most X^!^\ to A® A log‘®^*-^^A. 

We reduce the exponent of X from 1 or 5/6 to 1/2 by a more radical reorga- 
nization of the computation that lets us apply lattice reduction more efficiently. 
More generally, for each positive c (E Q we can hnd all cases of 0 < \cx^ — y^\ x 

in time Oc(A^/^ log‘®^*-^^A). All choices of c are essentially equivalent: we get from 
one to the other by scaling x, y and imposing congruence conditions on them. 
The most convenient choice of c turns out to be 4/3. We thus show how to solve 
0 < \Ax^ — 3j/^| ^ X] the cases relevant to Hall’s conjecture are those with i\x 
and 6|j/, when {'fx^ — 3j/^)/108 = {x/Z)'^ — (y/6)^. 
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We begin as in [H] by approximating x,y hy (multiples of) a square and a 
cube. Any positive integer x may be written uniquely as 

x — iC^ + rj with ?7,C e Zi,C> 0, e (-3C;3C]. (38) 

Then ^ 

(4^V3)'/" = 6C" + 3r?C+i^-^(^^) +0(1/C). (39) 

We thus write 

j/ = 6C^ + 3r?C + e (40) 

with ^ More precisely, if 

= /?C (41) 

Then /? £ (—3, 3], and — 3j/^| ^ a? if and only if 

e = ^-^/?" + 0(l/C). (42) 

At this point, Hall [H] imposes the assumption /? ^ We allow an arbitrary 

/? £ (~3,3] and approximate it within by one of evenly 

spaced points in that interval. Suppose, then, that b is one of those points. We 
approximate (42) by a linear combination of C, V — b(, and 1: 

6^ b b^ b'^ b b^ 

e = jC + 2 (^ - ^C) - + 0(1/0 = - jC + 2^ - 75 + 0(1/0- (43) 

We now assume that 7 5^ for instance by requiring that a? > A/4; repeating 

the computation with A replaced by A/4, A/16, A/64,. . . will then cover the 
entire range x < X, and if we can cover (A/4, A] in time 0(a?^0 log'^^^^A) then 
the same is true of [1,A]. Under the assumption x £ (A/4, A], we have the 
following constraints on 0^0- 

C « A^/2^ ?? - 6C « 1, (44) 

and 

e+ (45) 

We are thus in a familiar situation: we seek all the integral points in O(A^O) 
parallelepipeds, each of volume 0(1). The term b^/72 in (45) means that the 
parallelepipeds are no longer centered at the origin, but this causes no diflh- 
culty — indeed we already dealt with off-center parallelepipeds in the practi- 
cal implementation of our algorithm for Ending rational points near curves. So 
again we linearly transform each parallelepiped to a cube and obtain a lattice 
reduction problem; if these lattices were randomly distributed among three- 
dimensional lattices, we would almost certainly have only O(A^O) points to try, 
and would thus hud all solutions of 0 < \Ax^ — 3j/^| ^ x with a: < A in time 
0(Ai/Mog^(^)A). 
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In fact it turns out that in this case our lattices are not equidistributed: they 
all lie in a 2-dimensional subspace of the 5-dimensional moduli space of lattices 
in R®. This gives rise to both a minor annoyance and a major advantage. The 
bad news is that we cannot expect our lattices to have on average 0(1) vectors of 
norm ^ 1 ; but this annoyance is minor because the actual average is proportional 
to logdf and thus can be absorbed into the log'^^^^df factor. The good news is 
that we understand our special lattices well enough to actually prove results that 
are only heuristic for rational points near curves. 

The key is that in each case our lattice is a symmetric square of a lattice 
in R^. By this we mean the following. Recall that the symmetric square of a 
2-dimensional vector space V is the 3-dimensional vector space Sym^R consisting 
of symmetric tensors in R (g) R. Since Sym^R is dehned naturally in terms of R, 
any linear transformation of R yields a linear transformation of Sym^R. We thus 
have a homomorphism Sym^ : GL2 —t GL3. To give this map explicitly we choose 
a basis (ei, 62) for R, and use the basis (ei (g) ei, (ei (g) 62 -|- 62 (g) ei)/2, 62 G 62) 
for Sym^R. We then calculate that 

/ \ / P<1 \ 

Sym^ j ^ ^ = I 2pr ps + qr 2qs j . (46) 

\ rs s^ I 



Over any held, Sym^(SL2) is contained in the subgroup of SL3 preserving the 
discriminant form 4aia3 — on Sym^(R); if we worked over an algebraically 
closed held, that subgroup would coincide with Sym^(SL2). Now (44,45) mean 
that the column vector v — G R® satishes \\M(,v — m&|| ^ 1 where 

Uh (0, 0, —tfifTl) and 



/ 0 0 

Mf, := 0 1 -h 

VXi/2 _xR2^/2 N1/2^V4 



„ 2/^0 N - 1/4 



(47) 



This is why we went after — 3j/^ rather than pursuing x^ — directly: an 
analogous approach to x^ — y^ would yield a matrix that is still a symmet- 
ric square but with respect to a different basis, requiring a dehnition of Sym^ 
with fractional coefficients and complicating the lattice reduction. Note that the 
quadratic form 4^C — rf preserved by Mf, G Sym^(SL2) is already visible in (42). 

Our algorithm, then, is as follows. For each of our choices of b, 

calculate the matrix 

’= (a-W ^^4/2) 

with Mf, = Sym^ = W. Use lattice reduction to hud a matrix /Lf, G GL2(R) such 
that N(,K(, is as small as possible. Then 

Ml Sym^{NbKb) — Sym^W Sym^/bf, = Mb Sym^/bf,. (49) 

is small too. Let Lb — Sym^/Lf, G GL3(R). Then MbV — MlL'^^v. Find a box 
containing all w G R® such that ||M^w — Wf,|| ^ 1. For each w in the box. 
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compute V — LifW and check whether the resulting x, y satisfy x £ [X / A, X] and 
0 < \Ax^ — 3j/^| ^ X] if they do, output x (and check whether i\x and 6|j/ to 
determine whether this solution also yields a small value oix^ — y^). This is easier 
than our usual algorithm because we are reducing a lattice in rather than 
R®, which in our case amounts to calculating the continued fraction of hlX^I"^. 
Moreover, the computational cost of the algorithm can be bounded rigorously: 
Ml will only be large if hjX^I'^ is close to a rational number with numerator 
and denominator ^ X^!^ , and the eifect of such a close rational approximation 
is easy to determine. Summing over all rationals of height ^ X^!^ we hnd that 
the total number of candidate vectors u is ^ X^I'^logX , and thus that the 
computation takes time X) as claimed. 

Note that the X^I'^logX bound also has the following consequence: there 
are ^ X^I'^logX solutions oi \x^ — y'^\ ^ ^/x with x < X. Moreover, if C is 
large enough, we can deduce from this analysis that there are ^ X^^^ solutions 
of 0 < \x^ — y^ \ < C X with x £ \_X/2,X~\. More generally, we show that for each 
positive c £ (Q there exists C such that for each r £ R/R and d > 1 there are at 
most CdX^I'^logX solutions of \{cx — {y + r)\< dX with a:, j/ £ R and 
X < X; and, given c as above and any 9 £ [0, 1), there exists Co such that for 
any r £ R/R there are X^^^ solutions of \{cxY^I'^ ~ id + r} \ < C'oX~^^^ with 
x,y and x £ [OX, X], The constants C, Co depend eifectively on c, 9. These 
results improve considerably on results in this direction available from general 
exponential-sum techniques for proving uniform distribution mod 1. The detailed 
proofs of our claims in this paragraph will appear elsewhere. 

4.3 Numerical Results 

We have implemented our algorithm in a C program using 64-bit integer arith- 
metic, again replacing each 0{- ■ •) and ^ by explicit bounds, and searched for 
all solutions of 0 < \Ax^ — 3j/^| < 29)9)x^l'^ with 4 • 10® < x <Z ■ 10^®. The range 
X < 10^® was covered by a direct search, the overlap [4-10®, 10^®] being used as a 
check on the computation. The code was processed with an optimizing compiler 
and ran for three weeks during the summer of 1998 on a Sun Sparcstation Ul- 
tra 1. As a corollary we obtained all cases of 0 < — j/^| < ^\/x with x < 10^®. 

(With currently available hardware the same computation could easily hnish in 
a few days; with parallelization it should be feasible to reach 10^® at least.) The 
next table lists, for each of the 25 solutions of 0 < |a:® — j/® | < j\/x, the values of 
k — x^ — j/®, X, and r = x^!"^ /\k\. We need not list y, which is always the integer 
nearest to x^!"^. The explanation of the last two columns follows the table. 

The “GPZ” column indicates whether the solution was among the 13 listed 
in [GPZ]. These are the solutions with 1 < |fe| < 10®. Presumably the solution 
2®-32 = -1 is not on that list because the elliptic curve y^ — x^ + ^ was already 
known to have rank 0 so Gebel, Pet ho and Zimmer were not interested in it. 

The row is a new record, improving the previous record r by a factor of 
almost 10, whence the notation “!!”. Even row U^2, marked “!”, has r larger than 
the old record which is row Either of this suihces to refute Hall’s comment 
[H, p.l75], repeated in [GPZ], that r < 5 seems to hold in all cases. 
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*: Obtained from row by scaling [x, y, k) to {2"^x, 2^y, 2®fe). This reduces 
r by a factor of 32, but r = 46+ in row #1 is large enough that even r/32 still 
exceeds the threshold of our table. 

P{t): Birch’s polynomial family (33). This has r = 12/t + so the only 

values of f = 3 mod 6 that appear on the r > 1 list are t — ±3 and ±9. Already 
in [BCHS, p.69] the specializations t — ±3 are noted as “striking special cases” 
of (33). 

D: The hrst two cases of Danilov’s family (37). The appearance of the larger 
of these was a welcome check on our computation. 

Any threshold on r is of necessity arbitrary; the next solution has r just 
below our cutoff of 1: {x, k, r) = (16544006443618,4090263, 0.9944 . . .). 



# 


k 


X 


r 


GPZ? 


Comments 


1 


1641843 


5853886516781223 


46.60 




!! 


2 


30032270 


38115991067861271 


6.50 




! 


3 


-1090 


28187351 


4.87 


+ 




4 


-193234265 


810574762403977064 


4.66 






5 


-17 


5234 


4.26 


+ 


P(-3) 


6 


-225 


20114 


3.77 


+ 




7 


-24 


8158 


3.76 


+ 


P(3) 


8 


307 


939787 


3.16 


+ 




9 


207 


367806 


2.93 


+ 




10 


-28024 


3790689201 


2.20 


+ 




11 


-117073 


65589428378 


2.19 






12 


-4401169 


53197086958290 


1.66 






13 


105077952 


23415546067124892 


1.46 






14 


-1 


2 


1.41 






15 


-497218657 


471477085999389882 


1.38 






16 


-14668 


384242766 


1.34 


+ 


P(-9) 


17 


-14857 


390620082 


1.33 


+ 


m 


18 


-87002345 


12813608766102806 


1.30 






19 


2767769 


12438517260105 


1.27 






20 


-8569 


110781386 


1.23 


+ 




21 


5190544 


35495694227489 


1.15 






22 


-11492 


154319269 


1.08 


+ 




23 


-618 


421351 


1.05 


+ 




24 


548147655 


322001299796379844 


1.04 




D 


25 


-297 


93844 


1.03 


+ 


D 
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Abstract. We shall discuss the idea of finding all rational points on a 
curve C by first finding an associated collection of curves whose ratio- 
nal points cover those of C. This classical technique has recently been 
given a new lease of life by being combined with descent techniques on 
Jacobians of curves, Chabauty techniques, and the increased power of 
software to perform algebraic number theory. We shall survey recent ap- 
plications during the last 5 years which have used Chabauty techniques 
and covering collections of curves of genus 2 obtained from pullbacks 
along isogenies on their Jacobians. 



1 Introduction 

We consider a general curve of genus 2 defined over an number field K 

C-. Y^= F{X) = feX^ + + . . . + /o = Fi(V) . . . Fk{X), (1) 

where Fi{X), . . . , Fk{X) are the irreducible factors of F{X) over K; we assume 
that F{X) has no repeated roots and that /e 0 or /s 0. The intention is to 
describe, in a way accessible to a non-specialist, recent developments in Chabauty 
and covering techniques. These techniques all use essentially the same idea; we 
first find an Abelian variety A which maps to J ^ the Jacobian of C, under an 
isogeny (/). The pullbacks under (f> of a suitably chosen set of embeddings of C 
in give a collection of curves lying on A whose rational points cover those 
of C. Despite this rather geometric description, the mechanics of this, in the 
cases we shall consider, do not in fact require any difficult geometry. Provided 
the reader is prepared to take on faith a few standard results, the equations for 
the covering collections of curves can be obtained directly from that of C. 

In Section 2, we shall define the Jacobian of a curve of genus 2, and outline 
a few standard techniques for trying to find its rank. In Section 3, we describe 
Chabauty’s Theorem and, in particular, how it can be applied to the problem of 
finding the A-rational points on a curve of genus 2 defined over K; similar ideas 
can also be applied to an elliptic curve £ defined over a number field K, when 
one wants to find all points in £{K) subject to some arithmetic condition, such 
as the Q-rationality of the x-coordinate. In Sections 4,5, we describe the covering 
collections associated to various choices of isogeny, and give applications. Finally, 
in Section 6, we compare these techniques with a more classical approach using 
resultants. We shall try, in all sections, to provide sufficient detail that the non- 
specialist reader gains an impression of the techniques and difficulties involved. 
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We shall have in mind several motivating examples. The first of these concerns 
cycles of quadratic polynomials. Given a quadratic polynomial az^ + bz + c, 
with a, 6, c G Q and a yf 0, we say that 2 G Q is a point of exact period N 
if g^{z) = z and 5 "(z) yf 2 for all n < N. For example, 2 = 0 is a point of 
exact period 2 for 2 ^ — 1. It is easy to find such examples for N = 1,2,3, and it 
was shown in [19] that none exist for fV = 4. We shall later summarise the proof 
in [14] of the fact that none exist for N = 5 also. It remains an unsolved problem 
whether any examples exist for N > 6. Applying a linear transformation on z, 
we can assume that our quadratic is monic and has no linear term; that is, it 
is of the form g{z) = z"^ + c for some c G Q. Suppose that 2 is a point of exact 
period 5; then z, c must satisfy the curve {g^{z) — z)/{g{z) — z) = 0. This curve 
in z, c is of degree 30 and genus 14, but it has a quotient Ci of genus 2, derived 
in [14], given by 

Cl : = A® + 8A® + 22A-^ + 22A® + 5A^ + 6A + 1. (2) 

There are six obvious points oo^,(0,±l),(— 3,±1), where oo+joo” denote the 
points on the non-singular curve that lie over the singular point at infinity on C 
(for any curve (1) with /e yf 0 both oo’*' and oo“ are in C{K) when /e G (iC*)^). 
These six points do not have preimages corresponding to z, c G Q with z a 
point of exact period 5, and so the following Lemma gives a way of resolving the 
case N = 5. 

Lemma 1. Let C\ he as in (2). If Ci{Q) = {oo^, (0, ±1), (— 3, ±1)} then there 
is no quadratic polynomial in Q[z] with a rational point of exact period 5. 

Another application is to the equation 

a^ + b^ = c^, a^ + b^ + c^ = d^, a,b,c,d&Z. (3) 

There are the obvious solutions (3, 4, 5, 6), (4, 3, 5, 6), (1, 0, — 1, 0), (0, 1, — 1, 0), 
and we would like to show that these are all of them up to scalar multiplication. 
The first solution, the so-called “Nuptial Number of Plato”, is thought (see [29]) 
to be mentioned indirectly in Plato’s Republic, as being a special relationship 
between the 3-4-5 triangle (viewed at the time as the marriage triangle between 
the “male” number 3 and “female” number 4) and the first perfect number 6. 
It is shown in [29] that there are no other solutions, using 15 pages of lengthy 
but elementary resultant and congruence arguments. We shall give a different 
proof here, using the ideas of the next two sections. For the moment, we merely 
observe that, on dividing through by c, we get equations in the three affine vari- 
ables A = ajc,B = h/c,D = d/c. Furthermore, the solutions to -I- = 1 

can be parametrised as A = (1 — s^)/(l -I- s^),B = 2s/(l -I- s^). We substitute 
these into + B^ + \ = H®, multiply though by (1 -I- s^)®, and replace D 
by t = 0(1 -I- s^) to give the curve 

= 6s^ -k 8s® -k 2, (4) 

which is a plane quartic with a double point at s = — l,t = 0, and no other 
singularities, and so is of genus 2. Using a standard trick (see p.4 of [7]) which 
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involves mapping the double point to (0, 0) and then completing a square, we 
can birationally change variable to X = t/(l + s), F = 12s — 4 — t^/(l + s)^, 
which gives the equation + ?>2X^ — 32, with our four known solutions 

to (3) corresponding to oo*, (1,±1). 

Lemma 2. Let C 2 ■ = X^ + i2X^ - 32. // C 2 (Q) = {oo±,(l,±l)} then 

the only solutions to (3) are (3, 4, 5, 6), (4, 3, 5, 6), (1, 0, —1, 0), (0, 1, —1, 0) up to 
scalar multiples. 

Also of historical interest is Problem 17 of book VI of the Arabic manuscript 
of Arithmetica [22]. Diophantus poses a problem equivalent to finding a non- 
trivial rational point on the genus 2 curve 



Cs:Y^ = X^ + X^ + 1. (5) 

The related problem of finding all rational points has recently been solved by 
Wetherell [28], who showed that Cs^Q) = { 00 *, (0, ±1), (±1/2, ±9/8)} using 
Jacobians and covering techniques. We shall later give a sketch of the proof. This 
appears to be the only curve considered by Diophantus which has genus > 1. 

Another application, close to the heart of anyone who wants to construct ex- 
ercises for a calculus class, is that of Qr derived polynomials; that is, polynomials 
defined over Q, with all derivatives having all of their roots in Q. An example 
is f{x) = x(x — l)(a; — 8/3), f'(x) = (3x — 4/3)(a; — 2), f"(x) = 6x — 22/3. We 
say that a polynomial is of type Pmi,...,mr if it ii^is r distinct roots, and each nii 
is the multiplicity of the z-th root. Two Q-derived polynomials gi(x) and q 2 (x) 
are equivalent if q 2 (x) = rqi(sx ± t), for some constants r, s,t ^ Q, with r, s yf 0. 
The problem of classifying all Q-derived polynomials has been reduced in [5] to 
showing the following two conjectures. 



Conjecture 1. No polynomial of type pi, 1 , 1,1 is Q-derived. 

Conjecture 2. No polynomial of type P 3 ,i,i is Q-derived. 
Indeed, the following is shown in [5] . 



Theorem 1. If Conjectures 1 and 2 are true then all Q-derived polynomials are 
equivalent to one of 



‘(i-i). 



_ 1)(* _ ±^), - 1) (x - S’<2“’ + - - 12)(» + 2) 



V‘ 



{z - w - lS){Sw + z) 






for some n G Z+, v G Q, (w, z) G £o{Q), where Sq : = w{w — 6){w ± 18) is 

an elliptic curve of rank 1. 

For Conjecture 2, we let q{x) be a Q-derived polynomial of type P 3 ,i,i, which 
we may take to be in the form q{x) = x^(x — l)(a; — a), for some a G Q with 
a yf 0, 1. The discriminants of the quadratics q'”{x), q"{x)/x and q'{x)/x^, must 
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all be rational squares, and so must be their product. This implies that a satisfies 
(4a^ — 7a + 4)(9a^ — 12a + 9)(4a^ — 2a + 4) = 6^, for some b G Q. Using the 
transformation a = {X — 5)/ {X + 5), b = d>Y/{X + 3)^ gives the genus 2 curve 

Ca-.Y"^ = + 15) + 45) + 135). (6) 

The obvious points oo^, (±3, ±432) correspond to the illegal values a = 0, 1, oo, 
and so it is sufficient to show there are no others. 

Lemma 3. LcICa he as in (6). IfC 4 {Q) = {oo^, (±3, ±432)} then Conjecture 2 
is true. 

In Section 4, we shall sketch the proof in [16] that this is indeed all of C 4 (Q), 
and so now only Conjecture 1 (a surface) remains unsolved. 

A very recent result has been the solution in [17] of the “Serre curve” 

X> : ^ (7) 

Serre asks (p.67 of [21]) whether (x, y) = (±1, ±2), (±2, ±1) are the only x,y G Q 
satisfying (7). This curve is the only Fermat quartic of the type x^ + y^ = c, 
with c < 81, which cannot trivially be solved by local methods or by a map onto 
an elliptic curve of rank 0. It has gained some notoriety as being resistant to 
various methods of attack, but has finally succumbed to the general method we 
shall briefly mention in Section 5. 

The work of Bruin develops related ideas, which have been applied with great 
success to equations of the type x^ + y'^ = z'" . We shall mention two of these, 
and give an indication of the approach used. 

2 Preliminary Definitions 

At the risk of insulting the reader’s intelligence, we shall briefly summarise a few 
standard facts about elliptic curves. Consider the elliptic curve defined over K 

S : y'^ = G{x) = gsx^ + g 2 x'^ + gix + go = Gi{x) . . .Gk{x), (8) 

where G{x) has no repeated roots, go yf 0, and Gi(a;), . . . , Gk{x) are the irre- 
ducible factors of G{x) over K. Let oo denote the point at infinity, which we 
take to be the identity in the group S(K) of iL-rational points on £. The rules 
— {x, y) = (x, —y) and P+Q-\-R = oo P, Q, R are collinear, are sufficient to 
compute the group law on £{K), and the points of order 2 are of the form (x, 0), 
where a; G Lf is a root of G(x). The Mordell-Weil Theorem gives that £(X) is 
isomorphic to £(K)tor x Z”, where £(K)tor is the subgroup of £(K) consisting 
of points of finite order, and r is the rank of £{K). The finite group £{K)tor is 
normally found by using reduction maps modulo primes of good reduction. For 
each z G {1, . . . , fcj let be a root of Gi{x) and let Li = K{ai). 

Define the homomorphism 

y : £{K) (x,y) ^ [goix-ai), . . . , goix-Uk)], (9) 
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which has kernel 2£{K). Here, gs,{x — aj) is taken to be 1 when (x,y) = oo, 
and ~ “i) when x = Uj. If we let S' = {2,pi, . . .,Pm}, where pi,. . ,,pm 

are the rational primes of bad reduction, then the image of q is contained in- 
side the finite group M, consisting of those [c?i, . . . , dfc] such that all of the field 
extensions : Li, . . . , Lk{^/dk) '■ Lk are unramified outside of primes ly- 

ing over primes of S. Once M is determined, one eliminates members of M as 
potential members of ixa^q) by local (congruence) arguments. What remains is 
the 2-Selmer group, and one hopes that this is enough to determine the 2-rank 
of im(( 7 ), and hence that of £{K)/2£{K). If so, then one will have performed a 
successful complete 2-descent. On subtracting the 2-rank of £{K)tor/2£(K)tor, 
the remainder is the rank of £{K). A benefit of recent developments in algebraic 
number theory software, such as PARI/GP [1] and KASH [10], is that the above 
approach has become possible for elliptic curves defined over increasingly compli- 
cated number fields. Some of the methodology is described in [11], [20], [23], [24]; 
see also the program [4]. We mention here two such computations in the litera- 
ture ([15], [16], respectively) which will be relevant to later sections. 

Example 1. Let a satisfy -I- a -I- 1 = 0, and define over Q(a) the elliptic 
curves £\ : y^ = x(x^ -I- aa; -I- (a^ + 1)) and £2 '■ y^ = —ax(x^ -I- aa: -I- (a^ + 1)). 
Then £i(Q(a)) has rank 1, with generators given by (0,0), (—a, 1), where (0,0) 
is of order 2 and (— a, 1) is of infinite order. Also, £ 2 (Q(a)) has rank 0 and 
consists only of 00 and (0,0). 

Example 2. Let (3 = \/—15 and let £3 : y^ = 6(54 -|- 6/3) (—45a;^ -I- f){l3x + 1) 
and £4 : y^ = 6(9 -I- /3)(—45x^ + f){/3x + 1). Then TsIqIp)) has rank 1 and 
is generated by the 2-torsion point (— 1//3, 0) and the point (1/6-1- /3/30,24) of 
infinite order. Similarly, £4(Q(/3)) has rank 1 and is generated by the 2-torsion 
point {—1/(3, 0) and the point (—1/6 -I- /3/30, 9 -I- /3) of infinite order. 

Given a curve C of genus 2, as in (1) with /s ^ 0, we use 00 +, 00 “ as described 
after (2). When /e = 0 (and so /s yf 0), we let 00 denote the point at infinity, 
which is always in C{K). Following Ghapter 1 of [7], any member of J{K), the 
A'-rational points on the Jacobian, may be represented by a divisor of the form 
Pi -|- P 2 — 00 “'" — 00 “ , where Pi, P 2 are points on C and either Pi, P 2 are both 
A'-rational or Pi,P 2 are quadratic over K and conjugate. We shall abbreviate 
such a divisor by: {Pi, P 2 }. This representation gives a 1-1 correspondence with 
members of J{K), except that everything of the form {(A, Y), {X, — P)} must 
be identified into a single equivalence class O, which serves as the group identity 
in J{K). Note that -{(xi,yi),{x 2 ,y 2 )} = {{xi, -yi), {x 2 , -y 2 )}; furthermore 
|Pi,P 2 } -I- {Qi,Q 2 } + {Ri,R 2 } = O if and only if there exists T{X) of de- 
gree < 3 such that Y = T{X) meets C at Pi, P 2 , Qi, Q 2 , Ri, R 2 - These two 
rules are sufficient for computing the group law on £L{K). Glearly, an element 
of order 2 in J{K) is given by |(Ai, 0), (A 2 , 0)}, where Ai,A 2 are the roots 
of quadratic Q{X) defined over K, satisfying Q{X)\F{X). The Mordell-Weil 
Theorem gives that J{K) is isomorphic to J{K)tor x Z’’, where J{K)tor is the 
subgroup of J{K) consisting of points of finite order, and r is the rank of J{K). 
The finite group j7(A")tor is normally found by using reduction maps modulo 
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primes of good reduction. For each z S {1, . . . , fc} let Oi be a root of Fi{x) and 
let Li = K{ai). When Jq yf 0, we define the homomorphism 

Az : J{K) ^ [Lt/{Lt)^ X ... X Ll/{Ll)^)/~, 

• {(^Ij ^l)) (^ 2 , Y2)} [(-^1 — Ol)(-^2 — 0:2)5 ■ ■ ■ 5 (-^1 — Oik){X 2 — Ofc)], 

where the equivalence relation ~ is defined by 

[oi, . . . , afc]~[6i, . . ., &fc] ai = wbi, ... ,Qk = wbk, ioT some w € K* . (11) 

The interpretations of W — aj in special cases where {Xi,Yi) is a point at 
infinity, or Xi = aj, are as described immediately after (9). Either 2J{K) is 
the kernel of q or it has index 2 in the kernel of q (see [14]). The image of q 
is contained inside a finite group M, which is as described above for elliptic 
curves. Once M is determined, one proceeds in a similar manner to the complete 
2-descent for elliptic curves described above, and hopes to find [26] the 2-rank of 
J{K) j2J (K). There is some extra finesse here in determining the whether or not 
the kernel of q is 2f7 (K), and in the interpretation of the local information; there 
is also the potential for difficult computations in number fields of higher degree 
over the ground field than for elliptic curves. When /e = 0, the relation ~ can be 
removed, and the mechanics become more similar to that of complete 2-descent 
on an elliptic curve. As with elliptic curves, the final step is to subtract the 2-rank 
of j7(A')tor/2f7(A')tor from that of J{K)f2J{K) to obtain the rank of J{K). 
Recent developments in canonical heights and infinite descent ([13], [25], [27]) also 
allow actual generators for {K) to be computed in many cases. We mention 
here three ranks computed in the literature ([14], [28], [16], respectively), which 
we shall require later. Only the first of these is a genuine genus 2 computation, 
the other three ranks being computable via maps to elliptic curves. 

Example 3. Let Ci : = X^ + -h 22A^ -h 22A^ -h +QX +1, as 

in (2), with Jacobian J\. Then f7i(Q)tor = {O}; the rank o/f7i(Q) is 1, and it 
is generated by {00+, 00+}. 

Example 4. Let C2 : = A® -k 32A3 - 32, as in Lemma 2, with Jacobian J2. 

Then l72(Q)tor = {C’j {oo"*") oo~}}/ the rank of J2{Q) is 1, and it is 

generated by l72(Q)tor and {(1, l),oo“*'}. 

Example 5. Let C3 : Y^ = A® + A^ + 1, with Jacobian J3. Then l73(Q)tor = 
{O}; the rank ofJ'^^Q) is 2, and it is generated by {(0, 1), (0, 1)} and {(0, 1), 00+}. 

Example 6. Let C4 : = Fi(A)F2(A)F3(A), with Jacobian J4, where: 

Fi(A) = A2 -k 15, F2(A) = A2 -k 45, F^iX) = A^ -k 135, 
and let ai,(3i be the roots of Gi{X) for 1 < z < 3. Then 

J4(Q)tor = {O, {(Ol, 0), (A, 0)}, {(«2, 0), {P2, 0)}, {(«3, 0), (/?3, O)}}/ 
the rank o/f74(Q) is 2, and it is generated by the 2-torsion above, together with 
{oo'*', oo+l and {(3, 432), 00+}. 
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3 Chabauty’s Theorem 

Let E be an elliptic curve, as in (8), defined over a number field K = Q(a) of 
degree d. We shall consider the problem of trying to find all 

(x,y) G £(Q(of)) with a; G Q. (12) 

Imitating Chapter IV of [24] (with the difference that our equations in- 
clude 33, the coefficient of a;^), we introduce the variables s = —xfy, w = —Ify. 
Then w = g 3 S^+g 2 S^w+gisw^+gow^, and recursive substitution gives w = w(s), 
a power series in the local parameter s, with initial term gss^. Then 1/a; = w(s) / s 
is a power series 

-(s) = 53(5^ + 525^^ + (5153 + 52 )s® + 0(s®)) G Z[go,5i,52,53][[s]]- (13) 

X 

If (xo,yo) is another point on then the x-coordinate of (xo,yo) + {^^v) is a 
power series 

a;-coord of ( (a;o, yo) + {x, y)) = a;o + 2yos + (Bgaa^o + “^ 92 X 0 + 3i)s^ + O(s^) 

G ^go, 9i, 92, g3,xo,yo][[s]]. 

(14) 

If (s, w(s)), (t, w(t)) are two points in s-w coordinates then the s-coordinate of 
the sum can be written as T{s, t) G Z[go, 9i, 92 , 53] [[S7 1]]> the formal group. There 
are then power series 

log(t) = t+ + 1(52 + 25153)^® + 0 {f) G Q[5o,5i,52,53][W], (15) 

exp(t) = t- ^92t^+ ^(252 - 65153)^® + 0 {f) G Q[5 o, 5 i, 52,53 ][W], ( 16 ) 

satisfying log(lF(s, t)) = log(s) + log(t), lF(exp(s), exp(t)) =exp(s-|-t). In either 
power series, the denominator of the coefficient of divides k\. 

We now suppose that the rank r of £(Q(a)) is less than d = [Q(a) : Q], and 
that we have found generators for £(Q(a)): 

f(Q(a)) = (£(Q(a))tor,Pi,...,Pr). (17) 

Suppose that p is an odd prime such that jajp = 1, Q(a) is unramified at p, 
£ has good reduction at p, [Qp(a) : Qp] = [Q(a) : Q] = d, and |5i|p < 1, for 
i = 1, . . . , 3. These restrictions on p (which cannot be satisfied for some choices 
of a) are only for the sake of simplifying the exposition. Let a, £, P\, ... ,Pr 
represent, respectively, the reductions mod p of a, E, P\, . . . , Pr. Further define 
mi,Qi,Xi,yi,s^^'> by 

mi = order of Pi in £(Fp(d)), Qi = rmPi = (xi,yi), = -Xi/yi, (18) 

so that each Qi G £(Q(a)) is in the kernel of the reduction map from £(Q(a)) 
to £(Fp(d)), giving |s^*^|p < p~^. Now, let 5 be a set (which must be finite) of 
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representatives of £(Q(a)) modulo (Qi, . . . , Qr), so that every P e £(Q(a)) can 
be written uniquely in the form 



P — >5 + n\Q\ + . . . + TlrQri (19) 

for some S € S and ni,...rir G Z. We can now express the s-coordinate of 
niQi + . . . + rirQr, using (15), (16), as: exp(mlog(s^^)) + . . . + nrlog(s^’’^)), which 
is a power series in ni, . . . , Ur- Substituting this power series for s in (13) when 
S = oo, and in (14) when S = (xq, jjo) ^ oo gives 

0s(m, ...,rir) = xs(S + niQi + . . . + UrQr) G Zp[a][[ni, . ..,rir]], (20) 

where x$ means a;-coordinate, when S' yf oo, and l/a;-coordinate when S = oo. 
It is clear, from the standard estimate |fc!|p > that the coefficient 

of . . . nj?’’ is in Zp[a], and converges to 0 as fci + . . . + ^ oo. Splitting 9s 

into its components 

9s = ^a+. . .+9^g~^'^ , each 9g\ni, . . .Ur) G Zp[[m, . . . , n^]], (21) 

we obtain power series satisfying 

(x-coord of P) G Q ^ 4^^ = ■ ■ ■ = = 0- (22) 

We now make use of the following theorem (p.62 of [6]). 

Theorem 2. (Strassmann). Let 9(X) = cq + ciX + . . . G Zp[[X]] satisfy cj 0 
in Tip. Define £ uniquely by: |c^|p ^ |cj|p for all j ^ 0, and |c^|p > |cj|p for 
all j > £. Then there are at most £ values of x G Tp such that 9{x) = 0. 

When r, the rank of S(Q(a)), is 1 (as will be the case in the following ex- 
amples), and d = [Q(a) : Q] > 1, then we can apply Strassmann’s Theorem 
to bound, for example, the number of roots of 0g^(ni). In view of (22), sum- 
ming these bounds over all S G S gives an upper bound on the total number 
of (x, y) satisfying (12), which we hope to be the number of known such (x, y). 
When r > 1 and r < d, we can in principle try to perform repeated appli- 
cations if the Weierstrass Preparation Theorem (see p.108 of [6]) and resultant 
computations to derive univariate power series from d power series in r variables. 

Example 7. Let a,Si,S 2 be as in Example 1. Then the only (x,y) G £i(Q(a)) 
with X G Q are oo, (0, 0), ±(1/4, 1/8 — a/2 + o? /T). The only (x, y) G f2(Q(<a)) 
with X G Q are oo, (0, 0). 

Proof (see [15] for details): The result on £ 2 (Q(<a)) follows immediately from 
Example 1, since the rank is 0, and oo, (0, 0) are the only members of £ 2 (Q(q:)). 

For £i(Q(a)), let Pi = {—a, 1), p = 5, mi = 28; then 28Pi is in the kernel 
of reduction mod 5, but it is more efficient to take Qi = 14Pi ± (0, 0), which is 
also in the kernel of reduction mod 5. Let 



S = {kPi : -6 < fc < 7} U {(0, 0) ± fcPi : -6 < fc < 7}, 



(23) 
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SO that any P G £i(Q(a)) can be written as S' + niQi, for some S G S, rii G Z. 
Let us first consider S = — 2Pi = (1/4, 1/8 — a/2 + o? jP). Applying (15), (16), 
gives the s-coordinate of niQi as: 

exp(m log (s-coordinate of Qi)) = 5(21 -|- 15a -I- 21a^)ni (mod 5^). (24) 

Replacing (xo,yo) by (1/4, 1/8 — a/2 + a^/4) and s by (24) in (14) gives the 
a;-coordinate of — 2 Pi -|- niQi as: 

0 _ 2 Pi(ni) = 94 -I- 5(17a -I- 9a^)rii -I- 5^(2 -I- a -I- a^)n^ (mod 5^). (25) 

We may consider either 6 *i^ 2 Pi ^- 2 Pi> bue to the fact that the rank of S(Q(a)) 

is two less than [Q(a) : Q]. Taking 0 ^ 2 Pi(’^i) = 5 • 9 • rii -I- 5^ • (mod 5^), and 
applying Strassmann’s Theorem, gives that there is at most one root; but we 
know that rii = 0 is a root, since —2Pi -|- 0 • Qi has x-coordinate = 1/4 G Q. 
Hence ni = 0 is the only solution. Similarly, for S = oo,(0,0),2Pi we can 
show that rii = 0 is the value of rii for which S -I- niQi can have Q-rational 
a:-coordinate. For the remaining ten values of S G S, we find that 6g (ni) has 
constant term of 5-adic norm strictly greater than all subsequent coeffients; hence 
there are no roots in these cases. In summary, we have shown that oo, (0, 0), ±Pi 
are the only members of 5(Q(a)) with Q-rational x-coordinate, as required. □ 

A similar argument (see [16]), working mod 11^, shows the following. 

Example 8. Let /3 ,E^,Ej^ he as in Example 2. Then the only (x,y) G ifl3(Q(/3)) 
with X G Q are oo, ±(— 1/3, 12 -|- 12/3), ±(1/9, 12 -|- 4/3/3). Similarly, the only 
(x, y) G £ 4 (Q(/ 3 )) with x G Q are oo, ±(1/3, 12 — 4/3), ±(—1/9, 16/3). 

Now, consider a curve (1) of genus 2; suppose that it is defined over Q and 
that 17(Q) has rank 1. Given D = {(Ai,Fi), (A 2 ,± 2 )} G JL(Q), it is possible to 
describe a local parameter s = (si, S 2 ) given by 

51 = (0i(Ai, A 2 )Yi - gi(X 2 ,Xi)V 2 )(Xi - A 2 )/(±-o(Ai, A 2 ) - 2 riV 2 )^, 

5 2 = (0o(Ai, A 2 )±i - 0 o(A 2 , Ai)± 2 )(Ai - A 2 )/(±-o(Ai, A 2 ) - 2YiY2)^, 



where 

Po(Xi,X 2 ) = 2/0 ± /l(Ai ± A 2 ) ± 2/2(AiA2) ± / 3 (AiA 2 )(Ai ± A 2 ) 
±2/4(AiA2)2 ± /5(AiA2)2(Ai ± A 2 ) ± 2/e(AiA2)3, 

^i(Ai, A 2 ) = 2fo(Xi ± A 2 ) ± /iA2(3Ai ± A 2 ) ± 4/2(AiA|) 

+f3(X/Xi ± 3AiA|) ± U(2X/Xi + 2AiA|) 

±/5(3A/A| + AiA|) ± 4/6(A/A|), 

0 o(Ai, A2) = 4/0 ± /i(Ai ± 3A2) ± / 2 ( 2 AiA 2 ± 2 A|) ± M^X^Xl + A|) 

±4/4(AiA|) ± f5{X/Xl + 3AiA4) ± h{2X\X\ ± 2AiA|). 

The derivations of the above definitions are given in Chapter 7 of [7] . The reader 
can at least observe that si,S 2 will both be small when D is close to O. It 
is sufficient, in what follows, to accept on faith that s = (si,S 2 ) performs the 
same role on 17(Q) as s = —x/y does on an elliptic curve. Let Dq,D G J{Q), 
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with s = s{D) = (si{D), S 2 {D)) being the local parameter for D, and let 
Do + D = {{X[,Y(), {X 2 ,Y 2 )}. Then the group law on f7(Q) can be applied 
to find ■0^^^(s), power series in s, such that 

{1:X[+X!2-. X[X!2) = (4')(s) : : ^^^(s)), (27) 

where both sides should be viewed as projective triples. Associated to our lo- 
cal parameter is lF(s,t), the two-parameter formal group of f7(Q), the formal 
logarithm L = (Li, L 2 ) and exponential map E = {E\, E 2 ), given by 

Li(s) = Si -I- |(— 2/4sf -I- / 1 S 2 ) -I- . . . Ei{s) = Si -I- |(2/4sf — / 1 S 2 ) -I- . . . 

^ 2 ( 3 ) = S 2 -I- g( — 2 / 2 S 2 -I- / 5 S 1 ) + ■ ■ ■ £- 2 (s) = S 2 -I- g(2/2S2 — / 5 S 1 ) + ■ ■ ■ 

These satisfy L{E{s,t)) = L{s) + L{t) and E{s -|- t) = !F{E{s), E{t)). Now, 

suppose that f7(Q) = (J(Q)tor, T'l), and let p be a prime of good reduction. 
Let 77 and Di represent, respectively, the reductions modp of J and Di. Further 
define mi,Ei,s^^'> by 

mi = order of Di in 77(Fp) , Ei = miDi, = s(Di), (29) 

so that El G 77(Q) is in the kernel of the reduction map from 77(Q) to f7(Fp), 
giving Is^^^lp, |s 2 ^^|p < p~^. Now, let 5 be a set (which must be finite) of rep- 
resentatives of 77(Q) modulo {Ei), so that every D G 77(Q) can be written 
uniquely in the form 

D = S + mEi, (30) 

for some S G S and ni G Z. Now express s{D), using (28), as: exp(mlog(s(^))), 
which is a power series in ni. Substitute this power series for s in (27) and 
take Do = -S' to obtain 

= V’s^(exp(nilog(s(^)))) G Zp[[m]], fori =1,2, 3. (31) 

As with elliptic curves, the standard estimate |fc!|p > can be used 

to show that the coefficient of rii is in Zp, and converges to 0 as fc — > oo. 

So far, what we have achieved is to find a finite set of triples of power series, 
namely (0g ^(ni), ^(ni), 0g^^(rii)) for S G S, such that any D G 77(Q) has 

(1 : Ai -I- A 2 : A 1 A 2 ) equal to one of them. Now recall our original purpose, to 
find all of C(Q). The strategy is to embed the curve C into its Jacobian; we shall 
choose the map P 1 — > {P, P}, for any P G C(Q). This is not quite an injection, 
since any (A, 0) 1 -^ O; however, it is straightforward to find all Q-rational roots 
of the sextic E(X), and so all points (X, 0) G C(Q). Therefore, we can set these 
aside and concentrate on P = (A, V) with A yf 0, where P 1 — > {P, P} is injective. 
It is sufficient, then, to find all D G 77(Q) of the form D = {P, P}. Note that 
this implies Ai = A 2 , and so (Ai + A 2 )^ — 4 A 1 A 2 = 0, giving 



(32) 
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for some S G S - namely the S G S such that D = S mod (Ei). Our strategy, 
then, is to compute the power series in (32) and use Strassmann’s Theorem to 
find an upper bound on the number of possible ni. Adding these bounds together 
gives an upper bound on the number of (X,Y) € C(Q) with Y ^ 0, which we 
hope to be the same as the number of known points. We illustrate this with the 
following example from [14]. 

Example 9. LetCi he as in Example 3. T/ien Ci(Q) = {oo^, (0, ±1), (— 3, ±1)}. 

Proof: We already know from Example 3 that l7i(Q) has no nontrivial torsion 
and has rank 1, with f7i(Q) = {Di), where Di = {oo’*', oo+j. Let p = 3, which 
is a prime of good reduction, since the discriminant of the sextic is 2^^ • 3701. 
Let Di G 17(F3) denote the reduction of Di mod 3. The following lists the 
first few multiples of Di and Di. In the table, which is reproduced from [14], 
Po = (-^+ iVM, f and go = (-1 + f + |^/^), and 

Po and Qq are their conjugates over Q. 



n 


nD \ 


nD \ 


T 


0 


0 


1 


{00^, 00^} 


{00^, 00^} 


2 


1(0,1), (-3, 1)1 


{(0,1), (0,1)1 


3 


{(0,-1), 00 } 


{(0,-1), 00 } 


4 


{(0,-1), 00+} 


{(0,-1), 00+} 


5 


{(-3,1), 00-} 


{(0, 1),00“} 


6 


{(-3,1), 00+} 


{(0,1), 00+} 


7 


|(0,-1),(0,-1)| 


|(0,-1),(0,-1)| 


8 


{P,P} 


{00 , 00 } 


9 


1(0, -1), (-3, 1)1 


0 


10 


{ Q , Q } 


{00^, 00^} 


11 


{(-3,1), (-3, 1)1 


{(0,1), (0,1)1 



Table 1. The first 11 multiples of Di and Di. 



It is apparent that ±Pi, ±7Pi, ±llPi are all of the form {P,P}, and it is 
sufficient to show that no other member of Ji(Q) is of this form. Let Ei = 9Di, 
which is in the kernel of reduction mod 3 since 9Di = O, with corresponding 
local parameter (—9/14,426/49). Applying equation (28) we find that the local 
parameter of niEi is (36rii, 3ni+9nf) mod 3^. Any D G l7i(Q) can be written as 
D = S + niEi, for some S G S — {O, Di,2Di . . . , 8Di}. Consider, for example, 
S = 2Di. Using the group law to compute (27) mod 3^ a,t Dq = S = 2D\, and 
then substituting (36ni, 3ni + 9nf) for (si, S 2 ) gives (31) as 

= 25 + 15ni + 18n^ + 18nf (mod 3^), 
d^‘^) 2 Di{ni) = 6 + 24ni + 9n\ + 18n\ (mod 3^), 

0^3)2Di{ni) = 18ni + 18n1 (mod 3^), 



(33) 
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and so 9^2)2D^{niY — 40^1)2Di(?t^i)6'^3)2Di(?t^i) = 9 + ISn^ (mod 3^). Strass- 
mann’s Theorem tells us that there are at most two roots. In fact we know 
that n\ = ±1 are solutions, since 2Di + Ei = WDi = {(— 3, 1), (— 3, 1)} 
and 2Di — Ei = —7Di = {(0, 1), (0, 1)} are both of the form {P,P}. There- 
fore, Til = ±1 are the only rii € Z such that 2Di + riiEi is of the form {P, P}. 
Similar arguments show that: the only rii G Z such that Di + niEi is of 
the form {P,P} is rii = 0; the only rii G Z such that 7Di + niEi is of the 
form {P, P} are ni = 0, —2; the only ni G Z such that 8Pi -I- niEi is of the 
form {P, P} is ni = —1. For the remaining five S' G 5, Strassmann’s Theorem 
shows that S -I- riiPi is never of this form. Hence the upper bound on the order 
of Ci(Q) is six, and so oo^, (0, ±1), (—3, ±1) must give all of Ci(Q). □ 

Combining Lemma 1 and Example 9 gives us the result shown in [14] 

Theorem 3. There is no quadratic polynomial in Q[ 2 ] with a rational point of 
exact period 5. 

A similar argument, but using the prime p = 43, shows that C 2 (Q) = 
{oo^, (1,±1)}, where C 2 is as in (3) and Example 4 (which showed that J 2 {Q) 
has rank 1). In view of Lemma 2, this gives a new proof of the result originally 
shown in [29] by an elaborate set of resultant and congruence arguments. 

Theorem 4. The only integer solutions to of + 9"^ = + b^ + = d? are 

(3, 4, 5, 6), (4, 3, 5, 6), (1, 0, —1, 0), (0, 1, —1, 0) up to scalar multiplies. 

Both of the above examples are special cases of the following theorem of 
Chabauty [8]. 

Theorem 5. Let C he a curve of genus g defined over a number field K, whose 
Jacobian has Mordell-Weil rank ^ g — 1. Then C has only finitely many K- 
rational points. 

Apparent from the above examples is the similarity between the strategy for 
finding all {x, y) G £{K) with x G Q, where S is an elliptic curve, [iL : Q] = 2, 
and S{K) has rank 1 (sometimes called “Elliptic Curve Chabauty”), and that 
for finding C(Q), where C is a curve of genus 2 and f7(Q) has rank 1. In each 
case, S{K) or f7(Q), the group law is locally described by a 2-parameter system 
over Q, and an arithmetic condition, x € Q or Xi = A 2 , gives a power series in 
one variable ni. In general the local methods for finding all (x,y) G £{K) with 
X G Q, where £ is an elliptic curve, [iL : Q] = g, and £(K) has rank less than g, 
will be similar to those for finding C(Q), where C is a curve of genus g and f7(Q) 
has rank less than g. Sometimes one can even choose between either of these to 
solve the same problem. The work done in Example 1 turns out to be equivalent 
to showing Pi(Q) = { 00 , (0, ±1)} and p 2 (Q) = {oo}> where 

Pi : t^ = (s^ — 2s^ — 8s -|- l)(s^ -|- s -|- 1), /'q/i'i 

p 2 '.tf = (s^ — 8s — 4) (s^ -I- s^ -I- 1), 

both of genus 3. The derivation of Pi, P 2 will be made clear in the next section. 

We conclude this section with the result in [2], which also makes use of 
Chabauty’s Theorem. 
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Theorem 6. The only x,y, z € Z with (x, y, z) = 1, satisfying + y® = 2 ® are 
(±1, 0, 1), (0, ±1, 1) and (±1549034, ±33, 15613). 

The proof uses a parametrisation of ± = z® to obtain a covering of the 

solutions by the Q-rational points on five curves of genus 2. Two of these can 
be resolved by maps to elliptic curves. The remaining three all have 17(Q) of 
rank 1, and an argument similar to that used in the above examples can be used 
to find the rational points on each of them. 

We should also mention that it is also possible to use differentials instead 
of the formal group as way of applying Chabauty’s Theorem. This approach 
is described, for example, in [28]. For other work on Chabauty’s Theorem, see 
also [9], [12], [18]. 



4 Coverings of Bielliptic Curves 



We shall suppose, in this section, that our curve of genus 2 is defined over Q and 
has a Q-rational point, which has been mapped to infinity. Suppose also that 
there are only quadratic terms in X. 

C :Y^ = where G{x) = (x — ei)(x — e 2 )(x — € 3 ). (35) 

The map X 1 -^ —X swaps roots of the sextic of (35) in pairs, and the func- 
tion X = X^ is invariant under this map. There are then maps (X, V) 1 -^ (X^, V) 
and (X, Y) i-^- (1/X^, Y/X^) from C to the elliptic curves 

5“ : ^ = (a; - ei)(a; - e2){x - 63), 

= 2®G(1/^) = (— ei^± 1)(— ei^± 1)(— C32± 1), 



respectively. As in [28], these induce isogenies (j)\ : A\ ^ J and (()( : J — > Ai, 
where Ai = 5“ x 



(j)i : [(a;,y),(x,y)] {(Vi,y),(-Vi,y)}±{(^,^),(-- 

: {(Ai, w), (A 2 , ±2)} - [{XlY,) + (A|, ±2), (i^, ^) ± ( 



± ^)j 

, /T’ J ’ 



VI 

1 



Y 2 



)]■ 



(37) 

Both of 4>i, (p'l have kernels of order 4, and 4>io4>[ both give multiplication 

by 2 maps. There is furthermore an injective homomorphism (a special case 
of [20]): 



Ml : — Lt/{Lt)^ X L*/{L*)^ x L*/{L*)^ 

where : {(Ai,ri), (^ 2 ,^ 2 )} (A^ - ej)(A| - e^), for j = 1,2,3, 

(38) 

and where Li = Q(ci) for z = 1, 2, 3. This map is analogous to the map (x, y) ^ x 
used to perform descent via 2-isogeny on an elliptic curve y^ = x(x^ + ax + b) 
(see p.302 of [24]). 

Suppose that, after performing a descent, we have determined the set 



J(Q)/<(.i(Ai(Q)) = {Gi,...,G„}. 



(39) 
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Let {X, Y) G C(Q). Then {(X, V), oo+} = Di in (^i(Q)), for some 1 < 

i < m, and so V), oo+} = for j = 1,2,3, which is the same 

as {X'^ — 6 j) = in L* l{L*)'^ for j = 1, 2, 3. Since also G{X'^) is a square 

by (35), we have 

= (40) 

which is a curve of genus 1 defined over Lj (note that the right hand side is 
a quartic polynomial in X, after cancelling X'^ — ej). Multiplying both sides 
by X^, we see that the variables ytj = XVij and x = X^ satisfy 

ylj = /^^xG(x)/(x - €j), (41) 

an elliptic curve isogenous to the Jacobian of (40). We now have a strategy for 
trying to find the Q-rational points on the curve C in (35), even when J7(Q) has 
rank at least 2. Namely, for each i, one tries to find all {x, yij) on (41) using the 
techniques at the beginning of Section 3. The following was proved first in [28] 
and then [15]. The proof we sketch here is a blend of those two proofs. 

Theorem 7. Let C 3 : Y'^ = X^ + X"^ + 1, the Diophantus curve of (5) and 
Example 5. Then C 3 (Q) = { 00 ^, (0, ±1), (±1/2, ±9/8)}. 

Proof We take ci = a where ± a ± 1 = 0, and note that G{x) = ± 

a; ± 1 = (a; — a)(a;^ ± aa; ± (a^ ± 1)). From Example 5 we know that J 3 (Q) 
has rank 2 and is generated by {(0, 1), (0, 1)} and {(0, 1), 00 +}. We first note 
that {(0, 1), (0, 1)1 is trivial in ± 3 (Q)/</i (^i(Q)), as can be seen either by ap- 
plying (37) to get {(0, 1), (0, 1)1 = (/i([(0, l),oo]), or by applying (38) to get 
^i({(0, 1), (0, 1)1) = [1,1,1]. Applying (38) also gives that {(0,1), 00 +} yf O 
in ± 3 (Q)//'i(Ai(Q)). We conclude that ± 3 (Q)/(/i (Ai(Q)) has exactly two mem- 
bers: Di = O and D 2 = {(0, 1), 00 +}. 

Let (X,Y) G Cs(Q). Then {(A,V),oo+} = A or D 2 in J 3 (Q)/</i (Ai(Q)) . 
Applying (41) gives that a; = G Q satisfies one of the equations 

yf^i = x{x'^ + ax + {a'^ + l)), 

= — aa;(a;^ ± aa; ± (a^ ± 1)) ' ' 

We know from Example 7 that the only possible x G Q are x = 00 , 0, 1/4, and 
so any (A, Y) G Cs(Q) must satisfy A = 00 , 0, ±1/2, as required. □ 

As an alternative, note that if {(A, A), 00 +} = Ei = O in J 3 (Q)/</>i(Ai(Q)) 
then {(A, A), 00 +} = c/>i([Pa, J?h]) for some Pa G ■£“(Q), Pb G ■f^'’(Q)- Taking 
of both sides gives [(A^, A) ± 00 , (1/A^, A/A^) ± (0, 1)] = [2Pa, 2Pf\. Let s be 
the a;-coordinate of Pa, and let [2] a denote the a;-coordinate duplication map 
on Then 



A^ = [2]a(s) = {s* - 2s^ _ 8s -h l)/4(s3 ± s ± 1). (43) 

Letting t = 2(s^ ± s ± 1)A gives the model Ai in (34). 

Similarly, if {(A, A), 00 +} = D 2 = {(0,1), 00 +} in ± 3 (Q)/</i (Ai(Q)) then 
{(A, A), 00+1 - D 2 = {(A, A), (0,-1)} = O in J 3 (Q)/</i (Ai(Q)) , so that 
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{(X, y),(0,-l)} = S'h]) for some Sa € St G Taking (j)\ 

of both sides gives [{X'^ ,Y) + (0, — 1), (1/X^, + oo] = [25a, 25'^]. Let s 

be the x-coordinate of Sb^ and let [2]t denote the x-coordinate duplication map 
on 5^. Then 



1/^2 = [2]h(s) = (s^ - 8s - 4)/4 (s 3 + g2 ly ( 44 ) 

Letting t = 2(s^ + s^ + \)/X gives the model F 2 in (34). One can either, as we 
have done above, find all points the curves (42) with Q-rational x-coordinate; 
or, as in [28], one can find all members of Fi(Q), F 2 (Q). 

The underlying geometry is described in [28]. Each Di corresponds to an 
embedding of C into its Jacobian, given by P 1 — > {P, 00 +} — Di. If the Di give a 
complete set of representatives for J(Q)/(/)i(7li(Q)), then every member of C(Q) 
will be ‘hit’ by (()i(^(Q)) via one of these embeddings. It is therefore sufficient 
to find each Pi(Q), where T>i is the pullback of the embedded curve. Each T>i is 
a curve of genus 5 lying on A. and it has a hyperelliptic genus 3 quotient Ti. In 
our example, these are the Pi,p 2 of (34). Furthermore, the Jacobians of Pi,p 2 
are isogenous to the Weil restriction of scalars from Q(a) to Q of the curves 
in (42). 

If we try solve C 4 : = (Jf^ + 15) (Jf^ + 45) (Jf^ + 135) of (6) by the 

same technique, a problem arises. Here, e\ = — 15,62 = — 45,63 = —135, and 
every elliptic curve given by (41) is defined over Q. This means that, if the 
method is to work, for every i at least one curve (41) for j = 1,2 or 3 has to 
have rank 0. Applying (38) to the generators of J 74 (Q) given in Example 6, we 
find that the torsion group and {oo’*', oo+j are all trivial in J 4 (Q)/(()i (Ai(Q)) . 
Hence J 4 (Q)/(()i (Ai(Q)) just consists of the two elements O and {(3, 432), 00 +}. 
Let (X,Y) G C 4 (Q). Then {(A, E), oo"*"} is equal to either Di = O or D 2 = 
{(3, 432), 00 +} in J 4 (Q)/(/)i(.Ai(Q)). Consider first the case {(A, E), 00 +} = 
Pi = O in J 4 (Q)/(/)i(Ai(Q)). Then, using (41), we know that x = A^ satis- 
fies yl i = x(x -I- 45)(x -I- 135), for some j /14 G Q. This is an elliptic curve of 
rank 0 over Q, which has only the 2-torsion points with x = 00 , 0, —45, —135. 
None of 0,-45,-135 are rational squares and so they do not correspond to 
points (A, E) G C(Q). 

The case {(A, E), 00 +} = D 2 = {(3, 432), 00 +} is more troublesome. Us- 
ing (41), we know that x = A^ satisfies ^ = 24x(x -I- 45)(x -I- 135), 2 = 

54x(x-|-15)(x-|-135) and g = 144x(x-|- 15)(x-|-45), for some 2 / 2 , 1 , 2 / 2 , 2 , 2 / 2,3 G Q. 
These are elliptic curves of ranks 2,1,1, respectively, over Q, and so they do 
not restrict x to a finite number of choices. At this point, we have not deter- 
mined C 4 (Q), but we have shown 

(A, E) G C 4 (Q) ^ {(A, E), 00 +} = {(3, 432), 00 +} in (Ai(Q)) . (45) 

For the curve C 4 , the map A i-^- —A is not the only way of permuting the roots 
of the sextic. The curve is a special case of 



E^ = (A^ - k){X^ - rk){X^ - r^k), r, fc G Q, 



(46) 
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which has the involution {X, Y) {—rk/X, rk\/—rk Y/X^). The functions U = 
{X+^/ —rk) / {—X+^/—rk) and V = {8\/—rk Y)/{X—\/—rk)^ are invariant, and 
(X, Y) 1 -^ (C/^, V), {X, Y) 1 -^ (1/Cf^, V/U^) are maps from (46) to the quotient 

= —2k{u + l) ((r + — 2(r^ — 6r + l)u + (r + 1)^) , (47) 



defined over Q. Viewing (47) as being defined over Q{\/—rk), let A 2 be its Weil- 
restriction over Q. The maps (X,Y) 1 — > {U‘^,V), (X,Y) 1 -^ (1/C7^, V/U^) induce 
isogenies (f )2 '■ A 2 ^ J and (P 2 ■ J ^ A 2 , analogous to (j)i of (37), where here J 
is the Jacobian of (46). There is also an injective homomorphism 






m)/MMQ)) 



M^'^{(Xi,Vi),(X2,V2)} 

M^"^{(Xi,Vi),(X2,V2)} 



QV(Q*)2 X K*l{K*f,:D ^ [^^(iJ), 
>(X2-rfc)(X|-rfc), 

> {Xi — \/k){Xi + rVk){X2 — Vk){X2 + rVk), 



(48) 

where K = Q{\/k). Suppose that, after performing a descent, we have deter- 
mined the set 



Jm/MMQ))={D[,...,D'J. (49) 



Let (X,Y) G Q. Then {(Jf, V), 00 +} = Di in J(Q)/(()i (xli(Q)) , for some 1 < 
i < n, and so ^ 2 '^^({(Jf, V), 00 +} = for j = 1, 2. By a similar argument 

to that used for (41), we can show (see [16] for details) that u = 2Xl{X'^ — rk) 
satisfies 



Vi = ^^ 2 \D'^^^ 2 \D'i)irku^ + l)((r - l)v^ u/2 + 1), (50) 



for some yi G K. If this is an elliptic curve of rank 1, then we can try to apply 
the Elliptic Curve Chabauty techniques described at the beginning of Section 3. 
For our curve C 4 of ( 6 ), a special case of (46) with r = 3, fc = —15, we apply (38) 
to the generators of J 74 (Q) given in Example 6 , and find that { 00 +, 00 +} is 
trivial in J 4 (Q)/<(' 2 (^ 2 (Q))- Hence J 4 (Q)/(() 2 (H 2 (Q)) just consists of the eight 
elements generated by the 2-torsion and {(3, 432), 00 +}, that is: = 0,D'2 = 

{{P, 0), {-P, 0)}, iJ' = {(7=45, 0), (-7=45, 0)}, D'^ = {(37 0), (-37 0)}, D', = 
{(3, 432), 00 +}, iJ' = D',+D' 2 ,D', = D',+D'^, D'^ = D'^+D'^. Let (X, V) G C 4 (Q). 
Then {(Jf, V), 00 +} = D[ in T 4 (Q)/<)' 2 (H 2 (Q)) for some 1 < z < 8 . Now, for 
z = 1, . . . , 4, U' = O in J 4 (Q)/(/)i(Hi(Q)), which has already been discounted 
by (45). For z = 6,7, one can use a straightforward 5-adic argument (see [16]) 
to show the nonexistence of zz G Q 5 , z/z G Q 5 (/J), and hence the nonexistence 
of zz G Q, z/i G Q(7, satisfying (50). 

In summary, if {X, Y) G C 4 {Q), where C 4 is as in ( 6 ), then {(Jf, V), 00 +} = U' 
in J 4 (Q )/(()2 (H 2 (Q)) for z = 5 or z = 8 . Therefore zz = 2X/{X‘^ — rk) = 2X/{X"^ + 
45) G Q satisfies (50) for z = 5 or z = 8 (with r = 3,k = —15); that is, it satisfies 
one of the two equations 



z/| = 6(54 -I- 67(— 45zz^ -I- 1)(/Jzz -I- 1), 
z/| = 6(9 -I- 7(~45 zz^ -I- 1)(/Jzz -I- 1), 



for some z /5 or K = Q{P) = Q(7— 15). We have already seen, in Example 8 , 
that the only zz G Q on either curve are zz = 00 , ±1/3, ±1/9. For zz = 00 , ±1/3, 
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there are no X S Q satisfying u = 2Xj{X‘^ + 45). For u = ±1/9, there are X = 
±3, ±15; however, substituting X = ±15 into + 15)(X^ + 45)(X^ + 135) 
gives 23328000, which is nonsquare, and so there is no {X, Y) G C(Q) with X = 
±15. This leaves X = ±3 as the only possible ^-coordinates of an affine {X, Y) G 
C(Q). This proves that C(Q) = {oo^, (±3,432)}. In view of Lemma 3 this proves 
Conjecture 2, as in [16]. 

Theorem 8. No polynomial of type P 3 ,i,i is Q-derived. 

A feature of the above proof is that covers via both (pi and <p 2 were required; 
neither the (p\ nor the p 2 information on its own is sufficient to determine C 4 (Q). 

5 Coverings of a General Curve of Genus 2 

The next two sections use ideas of Nils Bruin, as in [2], [3], and variations by 
Flynn and Wetherell, as in [15], [17]. Let C : Y'^ = F{X) = Fi{X) . . .Fk{X) be 
a curve of genus 2, as in (1). We shall not assume that C is of any of the special 
types in the last section, although we shall continue to assume that C has a 
Q-rational point that has been mapped to infinity. Let p, be the map on (K) 
defined in (10), and suppose, as usual, that we have found J{K)/2J{K). It is 
then straightforward to deduce J(iC)/ker(/i), which we list as 

J{K)/ker{^i) = {Di,...,D„}. (52) 

Let P = {X, Y) G C{K) so that {P, oo+j € J{K). Then, for some i G {1, . . . , n|, 
we must have ^({(A, T), oo+|) = fi{Di). Let G{X) be any polynomial of even 
degree such that G(x)|F(a;). Then there is an induced map 

e i 

Mo : J{K) ^ Ly{Llf : [^ %■)] ^ J] G(a;,)"L (53) 

1=1 1=1 

where denotes the smallest field containing K over which G{x) is defined. Is 
follows that 



q^{Di)G{x) G (±o)^ for all G(a;)|F(a;) with 2|deg(G(a;)). (54) 

Each choice of G therefore gives a hyperelliptic curve v'j q = q^{Di)G{x), de- 
fined over ± 0 ) on which there must be an L^-v&iional point with iC-rational 
a;-coordinate. When G{x) has degree 4, it may be that this is an elliptic curve 
whose rank over is less that \L^ : K], In such cases, the Elliptic Curve 
Chabauty techniques at the beginning of Section 3 can be applied. This idea has 
recently been applied in [17] to V ■. x'^ + y'^ = 17, the “Serre curve”, as in (7). 
This is a curve of genus 3 whose Jacobian has rank 6. It is shown on pp. 187-189 
of [7] that the rearrangement 

{n +{bx^ -Axy + by^)){n -{bx^ -Axy + by^)) = -2{2x^ - bxy + 2y^f (55) 
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can be used, together with a resultant argument, to show that it is sufficient to 
find all Q-rational points on the curve of genus 2 

Cg : = (9^2 - 28X + 18)(X^ + 12X + 2){X'^ - 2). (56) 

Specifically, if C 5 (Q) has no affine points, then I?(Q) has only the affine points 
(±1, ±2), (±2, ±1). Equations (7) and (56) have stubbornly resisted the tech- 
niques described in the last two sections, as well as the method of Dem’yanenko 
(see [21], p.67). However, [17] finally showed, using the ideas sketched above, 
that it is sufficient to find all points on an elliptic curve over Q(-\/2, vTz) 
with Q-rational a;-coordinate. This elliptic curve, which we do not reproduce 
here (see [17]) has rank 1 over Q(-\/2, -\/T7); the Elliptic Curve Chabauty tech- 
niques at the beginning of Section 3 can be applied to show that indeed Cg(Q) 
has no affine points, from which T>{Q) can be deduced, as in [17]. 

Theorem 9. The only x,y G Q satisfying + y'^ = 17 are (±1, ±2), (±2, ±1). 

The technique to obtain the genus 2 cover (56) generalises to other Fermat 
quartics x'^ + y'^ = c, and so the methods of [17] are potentially applicable to 
other nontrivial values of c; that is, to the cases where x^ -\- y^ = c cannot be 
trivially solved by a direct local argument or a map to a rank 0 elliptic curve. 
There are only four such cases with c < 300, namely: c = 17, 82, 97, 257. 



6 A Classical Approach via Resultants 

Given a curve such as 



Cq:Y^ = {X^ + \){X^ + 1), (57) 

one could, if desired, apply the techniques described above. Here, Je(Q) has 
rank 2, and one can find Je(Q)/2 Je(Q)) followed by a set of coverings curves 
as described in the last two sections. However, it is worth bearing in mind that 
more than enough techniques were available to deal with such a curve long before 
recent methods for finding l7(Q)/2f7(Q). Letting X = ajb, where a,b gZ and 
gcd(a, b) = 1, and multiplying through by 6®, we have that fg is an integer 
square, where / = of + b"^ and g = + 6^. Now, if d = gcd{f,g) then d 

divides g — {of — b'^)f = 26^ and g + — b'^)f = 2a^. Since gcd(a, b) = 1, this 

means that d|2 and so d = ±1, ±2. Combining this with the fact that fg is an 
integer square gives that, for some choice of d = ±1, ±2, both of df and dg are 
integer squares. Dividing dg through by b^ we have, in particular that d{X^ + 1) 
is a Q-rational square, for some choice of d = ±1,±2. The negative values 
of d give no such such X S M and so no X G Q. This means that (X, Y) G 
Ce(Q) satisfies Yf = X^ + 1 for some Yi G Q or Yf = 2{X‘^ + 1) for some 
I 2 G Q. Both of these are rank 0 elliptic curves over Q, the first having only 
the points 00 ^, (0, ±1) and the second having only the points (±1, ±2), defined 
over Q. We can therefore say that Ce(Q) = { 00 *, (0, ±1), (±1, ±2)}, without 
having done anything sophisticated. 
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In principle, this idea can be attempted even when F(X) is written as a 
product of factors not defined over the ground field. When F{X) is written 
as F{X) = Qi{X)Q 2 {X), where Qi{X) is a quadratic and Q 2 {X) is a quartic, 
then resultant arguments (similar to those above) give a finite number of curves 
of genus 1 of the form: = dQ 2 {X), defined over an extension field, which 

need to be considered. One can then hope to apply Elliptic Curve Chabauty 
to each of these, and solve for C(Q) without ever having been required to com- 
pute f7(Q)/2f7(Q). In [3], this strategy is used to solve the following Diophantine 
problem. 

Theorem 10. The only x,y, z G Z with (x, y, z) = 1 and xyz yf 0, satisfying 
x» + y3 = are (x, y, z) = (±1, 2, ±3), (±43, 96222, ±3004207). 

In the proof of this result, ten associated curves of genus 2 are found, as in 
Theorem 6. Of these, there are three difficult cases which required the technique 
outlined in this section, together with the Elliptic Curve Chabauty technique at 
the beginning of Section 3. It would also be possible to solve these three cases 
using the strategy in Section 5. It is, to some extent, a matter of taste. The 
resultant method in [3] bypasses the need to find f7(Q)/2f7(Q). On the other 
hand, an initial computation of f7(Q)/2f7(Q) is often a straightforward and 
efficient way of removing many of the curves y^ = dQ 2 {X) from consideration. 

The author thanks Nils Bruin, Bjorn Poonen and Michael Stoll for their 
helpful comments on an earlier draft of this manuscript. 
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Abstract. Lattices are regular arrangements of points in space, whose 
study appeared in the 19th century in both number theory and crystal- 
lography. The goal of lattice reduction is to find useful representations 
of lattices. A major breakthrough in that field occurred twenty years 
ago, with the appearance of Lovasz’s reduction algorithm, also known 
as LLL or L®. Lattice reduction algorithms have since proved invaluable 
in many areas of mathematics and computer science, especially in al- 
gorithmic number theory and cryptology. In this paper, we survey some 
applications of lattices to cryptology. We focus on recent developments of 
lattice reduction both in cryptography and cryptanalysis, which followed 
seminal works of Ajtai and Coppersmith. 



1 Introduction 

Lattices are discrete subgroups of M”. A lattice has infinitely many Z-bases, 
but some are more useful than others. The goal of lattice reduction is to find 
interesting lattice bases, such as bases consisting of reasonably short and almost 
orthogonal vectors. From the mathematical point of view, the history of lattice 
reduction goes back to the reduction theory of quadratic forms developed by 
Lagrange [71], Gauss [44], Hermite [55], Korkine and Zolotarev [67,68], among 
others, and to Minkowski’s geometry of numbers [85]. With the advent of al- 
gorithmic number theory, the subject had a revival around 1980 with Lenstra’s 
celebrated work on integer programming (see [74]), which was, among others, 
based on a novel but non-polynomial time^ lattice reduction technique. That 
algorithm inspired Lovasz to develop a polynomial-time algorithm that com- 
putes a so-called reduced basis of a lattice. It reached a final form in the seminal 
paper [73] where Lenstra, Lenstra and Lovasz applied it to factor rational poly- 
nomials in polynomial time (back then, a famous problem), from which the name 
LLL comes. Further refinements of the LLL algorithm were later proposed, no- 
tably by Schnorr [101,102]. 

Those algorithms have proved invaluable in many areas of mathematics and 
computer science (see [75,64,109,52,30,69]). In particular, their relevance to cryp- 
tology was immediately understood, and they were used to break schemes based 

^ The technique is however polynomial-time for fixed dimension, which was enough 
in [74]. 
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on the knapsack problem (see [99,23]), which were early alternatives to the RSA 
cryptosystem [100] . The success of reduction algorithms at breaking various cryp- 
tographic schemes over the past twenty years (see [61]) have arguably established 
lattice reduction techniques as the most popular tool in public-key cryptanaly- 
sis. As a matter of fact, applications of lattices to cryptology have been mainly 
negative. Interestingly, it was noticed in many cryptanalytic experiments that 
LLL, as well as other lattice reduction algorithms, behave much more nicely than 
what was expected from the worst-case proved bounds. This led to a common 
belief among cryptographers, that lattice reduction is an easy problem, at least 
in practice. 

That belief has recently been challenged by some exciting progress on the 
complexity of lattice problems, which originated in large part in two seminal 
papers written by Ajtai in 1996 and in 1997 respectively. Prior to 1996, little 
was known on the complexity of lattice problems. In his 1996 paper [3], Ajtai 
discovered a fascinating connection between the worst-case complexity and the 
average-case complexity of some well-known lattice problems. Such a connection 
is not known to hold for any other problem in NP believed to be outside P. 
In his 1997 paper [4], building on previous work by Adleman [2], Ajtai further 
proved the NP-hardness (under randomized reductions) of the most famous lat- 
tice problem, the shortest vector problem (SVP). The NP-hardness of SVP has 
been a long standing open problem. Ajtai’s breakthroughs initiated a series of 
new results on the complexity of lattice problems, which are nicely surveyed by 
Cai [24,25]. 

Those complexity results opened the door to positive applications in cryp- 
tology. Indeed, several cryptographic schemes based on the hardness of lattice 
problems were proposed shortly after Ajtai’s discoveries (see [5,49,56,26,83,41]). 
Some have been broken, while others seem to resist state-of-the-art attacks, for 
now. Those schemes attracted interest for at least two reasons: on the one hand, 
there are very few public-key cryptosystems based on problems different from 
integer factorization or the discrete logarithm problem, and on the other hand, 
some of those schemes offered encryption/decryption rates asymptotically higher 
than classical schemes. Besides, one of those schemes, by Ajtai and Dwork [5], 
enjoyed a surprising security proof based on worst-case (instead of average-case) 
hardness assumptions. 

Independently of those developments, there has been renewed cryptographic 
interest in lattice reduction, following a beautiful work by Coppersmith [32] in 
1996. Coppersmith showed, by means of lattice reduction, how to solve rigor- 
ously certain problems, apparently non-linear, related to the question of finding 
small roots of low-degree polynomial equations. In particular, this has led to 
surprising attacks on the celebrated RSA [100] cryptosystem in special settings 
such as low public or private exponent. Coppersmith’s results differ from “tradi- 
tional” applications of lattice reduction in cryptanalysis, where the underlying 
problem is already linear, and the attack often heuristic by requiring (at least) 
that current lattice reduction algorithms behave ideally, as opposed to what is 
theoretically guaranteed. The use of lattice reduction techniques to solve poly- 
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nomial equations goes back to the eighties [54,110]. The first result of that kind, 
the broadcast attack on low-exponent RSA due to Hastad [54] , can be viewed as 
a weaker version of Coppersmith’s theorem on univariate modular polynomial 
equations. 

The rest of the paper is organized as follows. In Section 2, we give basic 
definitions and results on lattices and their algorithmic problems. In Section 3, 
we survey an old topic of lattice reduction in cryptology, the well-known sub- 
set sum or knapsack problem. Subsequent sections cover more recent applica- 
tions. In Section 4, we discuss lattice-based cryptography, somehow a revival for 
knapsack-based cryptography. In Section 5, we review the only positive applica- 
tion known of the LLL algorithm in cryptology, related to the hidden number 
problem. In Section 6, we discuss developments on the problem of finding small 
roots of polynomial equations, inspired by Coppersmith’s discoveries in 1996. 
In Section 7, we survey the surprising links between lattice reduction, the RSA 
cryptosystem, and integer factorization. 

2 Lattice Problems 

2.1 Definitions 

Recall that a lattice is a discrete (additive) subgroup of M”. In particular, any 
subgroup of Z” is a lattice, and such lattices are called integer lattices. An 
equivalent definition is that a lattice consists of all integral linear combinations 
of a set of linearly independent vectors, that is, 

T = < ^ nihi I rii e Z 

I i=i 

where the b^’s are linearly independent over M. Such a set of vectors b^’s is called 
a lattice basis. All the bases have the same number dim(T) of elements, called 
the dimension (or rank) of the lattice. 

There are infinitely many lattice bases. Any two bases are related to each 
other by some unimodular matrix (integral matrix of determinant ±1), and 
therefore all the bases share the same Gram determinant deti<ij<ci(bi, bj). The 
volume vol(L) (or determinant) of the lattice is by definition the square root 
of that Gram determinant, thus corresponding to the d-dimensional volume of 
the parallelepiped spanned by the b^’s. In the important case of full-dimensional 
lattices where dim(L) = n, the volume is equal to the absolute value of the 
determinant of any lattice basis (hence the name determinant). If the lattice is 
further an integer lattice, then the volume is also equal to the index [Z” : L] of 
L in Z". 

Since a lattice is discrete, it has a shortest non-zero vector: the Euclidean 
norm of such a vector is called the lattice first minimum, denoted by Ai(L) or 
j|Tj|. Of course, one can use other norms as well : we will use ||T||oo to denote 
the first minimum for the infinity norm. More generally, for all 1 < i < dim(L), 
Minkowski’s i-th minimum \i{L) is defined as the minimum of maxi<j<i||vjjj 




Phong Q. Nguyen and Jacques Stern 



over all i linearly independent lattice vectors vi , . . . , G L. It will be convenient 
to define the lattice gap as the ratio \ 2 {L) / Xi(L) between the first two minima. 

Minkowski’s Convex Body Theorem guarantees the existence of short vec- 
tors in lattices: a careful application shows that any d-dimensional lattice L 
satisfies ||T||oo < vol(L)^/'^, which is obviously the best possible bound. It fol- 
lows that Ai(L) < •\/dvol(L)^/‘^, which is not optimal, but shows that the value 
Ai(L)/vol(T)^/‘^ is bounded when L runs over all d-dimensional lattices. The 
supremum of Ai(L)^/vol(L)^/‘^ is denoted by 7 ^, and called Hermite’s constant^ 
of dimension d, because Hermite was the first to establish its existence in the 
language of quadratic forms. The best asymptotic bounds known for Hermite’s 
constant are the following ones (see [84, Chapter II] for the lower bound, and [31, 
Chapter 9] for the upper bound): 



d 

27re 



log(Trd) 

27re 



+ 0(1) < 7d < 



1.744d 

27re 



( 1 + 0 ( 1 )). 



Minkowski proved more generally: 

Theorem 1 (Minkowski). For all d-dimensional lattice L and all r < d : 



l[HL) < 

i=l 



More information on lattice theory can be found in numerous textbooks, such 
as [53,108,76]. 



2.2 Algorithmic Problems 

In the rest of this section, we assume implicitly that lattices are rational lattices 
(lattices in Q”), and d will denote the lattice dimension. 

The most famous lattice problem is the shortest vector problem (SVP), which 
was apparently first stated by Dirichlet in 1842: given a basis of a lattice L, find 
V G L such that ||v|| = Ai(L). SVP 00 will denote the analogue for the infinity 
norm. One defines approximate short vector problems by asking a non-zero v G T 
with norm bounded by some approximation factor: ||v|| < /(d)Ai(L). 

The closest vector problem (CVP), also called the nearest lattice point prob- 
lem, is a non-homogeneous version of the shortest vector problem: given a lattice 
basis and a vector v G M”, find a lattice vector minimizing the distance to v. 
Again, one can define approximate versions. 

Another problem is the smallest basis problem (SBP), which has many vari- 
ants depending on the exact meaning of “smallest” . The variant currently in 
vogue (see [3,11]) is the following: find a lattice basis minimizing the maximum 
of the lengths of its elements. A more geometric variant asks instead to minimize 
the product of the lengths (see [52]). 



^ For historical reasons, Hermite’s constant refers to maxAi(L)^/vol(L)^^‘* and not 
max Ai (L) /vol(L)^^‘*. 
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2.3 Complexity Results 

We refer to Cai [24,25] for an up-to-date survey of complexity results. Ajtai [4] 
recently proved that SVP is NP-hard under randomized reductions. Miccian- 
cio [82,81] simplified and improved the result by showing that approximating 
SVP to within a factor < -\/2 is also NP-hard under randomized reductions. The 
NP-hardness of SVP under deterministic (Karp) reductions remains an open 
problem. 

CVP seems to be a more difficult problem. Goldreich et al. [50] recently 
noticed that CVP cannot be easier than SVP: given an oracle that approximates 
CVP to within a factor /(d), one can approximate SVP in polynomial time 
to within the same factor /(d). Reciprocally, Kannan proved in [64] that any 
algorithm approximating SVP to within a non-decreasing function /(d) can be 
used to approximate CVP to within d^/^/(d)^. CVP was shown to be NP-hard 
as early as in 1981 [40] (for a simplified proof, see [65]). Approximating CVP to 
within a quasi-polynomial factor 2*°® is NP-hard [6,38]. 

However, NP-hardness results for SVP and CVP have limits. Goldreich and 
Goldwasser [46] showed that approximating SVP or CVP to within i/d/0 (log d) 
is not NP-hard, unless the polynomial-time hierarchy collapses. 

Interestingly, SVP and CVP problems seem to be more difficult with the 
infinity norm. It was shown that SVPoo and CVPoo are NP-hard in 1981 [40]. 
In fact, approximating SVPoo/CVPqo to within an almost-polynomial factor 
^ 1 / log log d jg ]\jp_];iard [37]. On the other hand, Goldreich and Goldwasser [46] 
showed that approximating SVPoo/CVPqo to within d/0(logd) is not NP-hard, 
unless the polynomial-time hierarchy collapses. 

We will not discuss Ajtai’s worst-case/ average-case equivalence [3,27], which 
refers to special versions of SVP and SBP (see [24,25,11]) such as SVP when the 
lattice gap A 2 /A 1 is at least polynomial in the dimension. 



2.4 Algorithmic Results 

The main algorithmic results are surveyed in [75,64,109,52,30,69,24,97]. No poly- 
nomial-time algorithm is known for approximating either SVP, CVP or SBP to 
within a polynomial factor in the dimension d. In fact, the existence of such 
algorithms is an important open problem. The best polynomial time algorithms 
achieve only slightly subexponential factors, and are based on the LLL algo- 
rithm [73], which can approximate SVP and SBP. However, it should be empha- 
sized that these algorithms typically perform much better than is theoretically 
guaranteed, on instances of practical interest. Given as input any basis of a lattice 
L, LLL provably outputs in polynomial time a basis (bi, . . . , b^) satisfying : 

d 

||bi|| < 2(‘^-i)/W(L)i/‘^, Ijbill < 2(‘^-i)/2Ai(L) and ||bi]| < 
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Thus, LLL can approximate SVP to within Schnorr^ [101] improved the 

bound to 2‘^(‘^0ogiog<i) /iog<i)^ fact, he defined an LLL-based family of algo- 
rithms [101] (named BKZ for blockwise Korkine-Zolotarev) whose performances 
depend on a parameter called the blocksize. These algorithms use some kind of 
exhaustive search exponential in the blocksize. So far, the best reduction algo- 
rithms in practice are variants [104,105] of those BKZ-algorithms, which apply 
a heuristic to reduce exhaustive search. But little is known on the average-case 
(and even worst-case) complexity of reduction algorithms. 

Babai’s nearest plane algorithm [7] uses LLL to approximate CVP to within 
2*^/^, in polynomial time (see also [66]). Using Schnorr’s algorithm [101], this 
can be improved to 2 ‘^(‘^*^*°siog<i)^/iog<i)^ ^o Kannan’s link between CVP and 
SVP (see previous section) . In practice however, the best strategy seems to be the 
embedding method (see [49,90]), which uses the previous algorithms for SVP and 
a simple heuristic reduction from CVP to SVP. Namely, given a lattice basis 
(bi, . . .,bd) and a vector v e M", the embedding method builds the {d + 1)- 
dimensional lattice (in M”+^) spanned by the row vectors (bi,0) and (v, 1). It 
is hoped^ that a shortest vector of that lattice is of the form (v — u, 1) where 
u is a closest vector to v, in the original lattice . Depending on the lattice, one 
should choose a coefficient different than 1 in (v, 1). 

For exact SVP or CVP, the best algorithms known (in theory) are Kannan’s 
super-exponential algorithms [63,65], with running time 



3 Knapsacks 

Cryptology and lattices share a long history with the knapsack (also called subset 
sum) problem, a well-known NP-hard problem considered by Karp: given a set 
{oi, 02 , ... , o„} of positive integers and a sum s = Y^=i where Xi G {0, 1}, 
recover the Xj’s. 

In 1978, Merkle and Hellman[80] invented one of the first public-key cryp- 
tosystems, by converting some easy knapsacks into what they believed were 
hard knapsacks. It was basically the unique alternative to RSA until 1982, when 
Shamir [106] proposed an attack against the simplest version of the Merkle- 
Hellman scheme. Shamir used Lenstra’s integer programming algorithm [74] but, 
the same year, Adleman [1] showed how to use LLL instead, making experiments 
much easier. Brickell [21,22] later extended the attacks to the more general “it- 
erated” Merkle-Hellman scheme, and showed that Merkle-Hellman was insecure 
for all realistic parameters. The cryptanalysis of Merkle-Hellman schemes was 
the first application of lattice reduction in cryptology. 

Despite the failure of Merkle-Hellman cryptosystems, researchers continued 
to search for knapsack cryptosystems because such systems are very easy to 

® Schnorr’s result is usually cited in the literature as an approximation algorithm to 
within (1-l-e)” for any constant e > 0. However, Goldreich and Hastad noticed about 
a year ago that one can choose some e = o(l) and still have polynomial running time, 
for instance using the blocksize k — log d/ log log d in [101]. 

^ Note that there exist simple counter-examples (see for instance [81]). 
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implement and can attain very high encryption/decryption rates. But basically, 
all knapsack cryptosystems have been broken (for a survey, see [99]), either 
by specific (often lattice-based) attacks or by the low-density attacks. The last 
significant candidate to survive was the Chor-Rivest cryptosystem [29], broken 
by Vaudenay [112] in 1997 with algebraic (not lattice) methods. 



3.1 Low-Density Attacks 

We only mention some of the links between lattices and knapsacks. Note that 
Ajtai’s original proof [4] for the NP-hardness (under randomized reductions) of 
SVP used a connection between the subset sum problem and SVP. 

The knapsack density is defined as d = n/ maxi<i<„log 2 a^. The low-density 
attacks establish a reduction from the subset sum problem to the lattice short- 
est vector problem. The first low-density attack used the n-dimensional lat- 
tice L(ai, . . . , a„, s) in formed by the vectors (j/i, . . . , y„+i) such that 

yiQi -I- • • • -I- j/nOn = yn+is. Such a lattice can easily be built in polynomial 
time from the o^’s and s. It was proved by Lagarias and Odlyzko [70] that 
if d < 0.6463..., the target vector (a;i, . . . , cc„, 1) was the shortest vector of 
L(oi, . . . , a„, s) with high probability over the choice of the Oi’s. The proof relies 
on bounds [77] on the number of integer points in n-dimensional balls. Thus, 
if one has access to an SVP-oracle, one can solve most subset sum problems of 

density d < 0.6463 Coster et al. [34] later improved the connection between 

SVP and the knapsack problem. By using a simple variant of L{ai, . . . , a„, s), 
they showed that if d < 0.9408..., the knapsack problem can be reduced to 
a lattice shortest vector problem (in dimension n) with high probability. In a 
different context (polynomial interpolation in the presence of noise), another 
example of attack based on provable reduction to SVP appeared recently in [10]. 

In the light of recent results on the complexity of SVP, those reductions from 
knapsack to SVP may seem useless. Indeed, the NP-hardness of SVP under ran- 
domized reductions suggests that there is no polynomial-time algorithm that 
solves SVP. However, it turns out that in practice, one can hope that standard 
lattice reduction algorithms behave like SVP-oracles, up to reasonably high di- 
mensions. Experiments carried out in [70,104,105] show the effectiveness of such 
approach for solving low-density subset sums, up to n about the range of 100- 
200. It does not prove nor disprove that one can solve, in theory or in practice, 
low-density knapsacks with n over several hundreds. But it was sufficient to 
show that knapsack cryptography was impractical: indeed, the key size of knap- 
sack schemes grows in general at least quadratically with n, so that high values 
of n (as required by lattice attacks) are not practical. 

One might wonder whether those reductions can lead to provable polynomial- 
time algorithms for certain subset sums. Recall that LLL is an SVP-oracle when 
the lattice gap is exponential in the lattice dimension. For lattices used in knap- 
sack reductions, the gap increases as the knapsack density decreases, however the 
gap can be proved to be large enough only in extremely low density (see [42,43]). 
Hence, lattice methods to solve the subset sum problem are very heuristic. And 
lattice attacks against knapsack cryptosystems are somehow even more heuristic. 
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because the reductions from knapsack to SVP assume some (natural) property 
on the distribution of the weights at’s, which is in general not satisfied by knap- 
sacks arising from cryptosystems. 



3.2 The Orthogonal Lattice 

Recently, Nguyen and Stern proposed in [91] a natural generalization of the 
Lagarias-Odlyzko [70] lattices. More precisely, they defined for any integer lattice 
L in Z”, the orthogonal lattice L-^ as the set of integer vectors orthogonal to L, 
that is, the set of x e Z” such that the dot product (x, y) = 0 for all y G L. 
Note that the lattice has dimension n — dim(L), and can be computed in 
polynomial time from L (see [30] ) . Interestingly, the links between duality and 
orthogonality (see Martinet’s book [76, pages 34-35]) enable to prove that the 
volume of is equal to the volume of the intersection L of Z” with the linear 
span of L. Thus, if a lattice in Z” is low-dimensional, its orthogonal lattice is 
high-dimensional with a volume at most equal: the successive minima of the 
orthogonal lattice are likely to be much shorter than the ones of the original 
lattice. That property of orthogonal lattices has led to effective (though heuristic) 
lattice-based attacks on various cryptographic schemes [91,93,94,92,95]. We refer 
to [96,97] for more information. In particular, it was used in [95] to solve the 
hidden subset sum problem (used in [20]) in low density. The hidden subset sum 
problem was apparently a non-linear version of the subset sum problem: given 
M and n in N, and b\, ... ,bm G Zm, find ai, . . . , G Zm such that each bi is 
some subset sum modulo M of oi, . . . , a„. 

We sketch the solution of [95] to give a flavour of cryptanalyses based on 
orthogonal lattices. We first restate the hidden subset sum problem in terms of 
vectors. We are given an integer M, and a vector b = (6i, . . . , bm) G Z™ with 
entries in [0..M — 1] such that there exist integers a\, . . .,a„ G [0..M — 1], and 
vectors xi, . . . , x„ G Z'" with entries in {0, 1} satisfying: 

b = QfiXi -I- « 2 X 2 -I- • • • -I- OnXn (mod M). 

We want to determine the ai’s. There exists a vector k G Z™ such that: 
b = QfiXi -I- 02X2 -k • • • -k OnXn + Mk. 

Notice that if u in Z” is orthogonal to b, then Pu = ((u, xi), . . . , (u, x„), (u, k)) 
is orthogonal to the vector = (oi, . . . , a„, M). But is independent of m, 
and so is the n-dimensional lattice v^. On the other hand, as m grows for a 
fixed M, most of the vectors of any reduced basis of the (m — l)-dimensional 
lattice b-'- should get shorter and shorter, because they should have norm close 
to vol(b-^)^/*^™“^^ < vol(b)^/*^™“^^ = « {M -JmY / For such 

vectors u, the corresponding vectors p„ also get shorter and shorter. But if p„ 
gets smaller than Ai(v^) (which is independent of m), then it is actually zero, 
that is, u is orthogonal to all the Xj’s and k. Note that one expects Ai(v^) to 
be of the order of ||vq||^/” « (My^)^/”. 
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This suggests that if (ui, . . . , Um_i) is a sufficiently reduced basis of b-'^, then 
the first m— (n+ 1) vectors Ui, . . . , should heuristically be orthogonal 

to all the Xj’s and k. One cannot expect that more than m — (n + 1) vectors 
are orthogonal because the lattice spanned by the xj’s and k is likely to 
have dimension (n + 1). From the previous discussion, one can hope that the 
heuristic condition is satisfied when the density n/log(M) is very small (so 
that Ai(v^) is not too small), and m is sufficiently large. And if the heuristic 
condition is satisfied, the lattice is disclosed, because it is then equal to the 
orthogonal lattice (ui, . . . , Once is known, it is not difficult to 

recover (heuristically) the vectors xj’s by lattice reduction, because they are 
very short vectors. One eventually determines the coefficients aj’s from a linear 
modular system. The method is quite heuristic, but it works in practice for small 
parameters in low density (see [95] for more details). 

4 Lattice-Based Cryptography 

We review state-of-the-art results on the main lattice-based cryptosystems. To 
keep the presentation simple, descriptions of the schemes are intuitive, referring 
to the original papers for more details. Only one of these schemes (the GGH 
cryptosystem [49]) explicitly works with lattices. 

4.1 The Ajtai Dwork Cryptosystem 

Description. The Ajtai-Dwork cryptosystem [5] (AD) works in M”, with some 
finite precision depending on n. Its security is based on a variant of SVP. 

The private key is a uniformly chosen vector u in the n-dimensional unit 
ball. One then defines a distribution of points a in a large n-dimensional 
cube such that the dot product (a, u) is very close to Z. 

The public key is obtained by picking wi , . . . , w„, Vi , . . . , (where m = n^) 
independently at random from the distribution 7i„, subject to the constraint 
that the parallelepiped w spanned by the w^’s is not flat. Thus, the public key 
consists of a polynomial number of points close to a collection of parallel affine 
hyperplanes, which is kept secret. 

The scheme is mainly of theoretical purpose, as encryption is bit-by-bit. To 
encrypt a ’O’, one randomly selects b\, . . . ,bm in {0, 1}, and reduces 
modulo the parallelepiped w. The vector obtained is the ciphertext. The cipher- 
text of ’1’ is just a randomly chosen vector in the parallelepiped w. To decrypt 
a ciphertext x with the private key u, one computes r = (x, u). If r is suffi- 
ciently close to Z, then x is decrypted as ’O’, and otherwise as ’!’. Thus, an 
encryption of ’0’ will always be decrypted as ’O’, and an encryption of ’1’ has a 
small probability to be decrypted as ’O’. These decryption errors can be removed 
(see [48]). 

Security. The Ajtai-Dwork [5] cryptosystem received wide attention due to a 
surprising security proof based on worst-case assumptions. Indeed, it was shown 
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that any probabilistic algorithm distinguishing encryptions of a ’0’ from encryp- 
tions of a ’1’ with some polynomial advantage can be used to solve SVP in any 
n-dimensional lattice with gap A 2 /A 1 larger than n®. There is a converse, due to 
Nguyen and Stern [93]: one can decrypt in polynomial time with high probability, 
provided an oracle that approximates SVP to within or one that approxi- 

mates CVP to within It follows that the problem of decrypting ciphertexts 
is unlikely to be NP-hard, due to the result of Goldreich-Goldwasser [46] . 

Nguyen and Stern [93] further presented a heuristic attack to recover the 
secret key. Experiments suggest that the attack is likely to succeed up to at 
least n = 32. For such parameters, the system is already impractical, as the 
public key requires 20 megabytes and the ciphertext for each bit has bit-length 
6144. This shows that unless major improvements® are found, the Ajtai-Dwork 
cryptosystem is only of theoretical importance. 



Cryptanalysis Overview. At this point, the reader might wonder how lat- 
tices come into play, since the description of AD does not involve lattices. Any 
ciphertext of ’0’ is a sum of v^’s minus some integer linear combination of the 
Wi’s. Since the parallelepiped spanned by the w^’s is not too flat, the coeffi- 
cients of the linear combination are relatively small. On the other hand, any 
linear combination of the v^’s and the w^’s with small coefficients is close to 
the hidden hyperplanes. This enables to build a particular lattice of dimension 
n + m such that any ciphertext of ’0’ is in some sense close to the lattice, and 
reciprocally, any point sufficiently close to the lattice gives rise to a ciphertext 
of ’O’. Thus, one can decrypt ciphertexts provided an oracle that approximates 
GVP sufficiently well. The analogous version for SVP uses related ideas, but is 
technically more complicated. For more details, see [93]. 

The attack to recover the secret key can be described quite easily. One knows 
that each (v j , u) is close to some unknown integer Vj . It can be shown that any 
sufficiently short linear combination of the Vj’s give information on the Pi’s. 
More precisely, if A^v^ is sufficiently short and the A^’s are sufficiently small, 
then = 0 (because it is a too small integer). Note that the Pi’s are 

disclosed if enough such equations are found. And each Pi gives an approximate 
linear equation satisfied by the coefficients of the secret key u. Thus, one can 
compute a sufficiently good approximation of u from the Pi’s. To And the Pi’s, we 
produce many short combinations XiVi with small APs, using lattice reduc- 
tion. Heuristic arguments can justify that there exist enough such combinations. 
Experiments showed that the assumption was reasonable in practice. 

4.2 The Goldreich Goldwasser Halevi Gryptosystem 

The Goldreich-Goldwasser-Halevi cryptosystem [49] (GGH) can be viewed as 
a lattice-analog to the McEliece [78] cryptosystem based on algebraic coding 
theory. In both schemes, a ciphertext is the addition of a random noise vector 

® A variant of AD with less message expansion was proposed in [26] , however without 
any security proof. It mixes AD with a knapsack. 
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to a vector corresponding to the plaintext. The public key and the private key 
are two representations of the same object (a lattice for GGH, a linear code for 
McEliece). The private key has a particular structure allowing to cancel noise 
vectors up to a certain bound. However, the domains in which all these operations 
take place are quite different. 



Description. The GGH scheme works in Z”. The private key is a non-singular 
n X n integral matrix R, with very short row vectors® (entries polynomial in n) . 
The lattice L is the full-dimensional lattice in Z” spanned by the rows of R. 
The basis R is then transformed to a non-reduced basis B, which will be public. 
In the original scheme, B is the multiplication of R by sufficiently many small 
unimodular matrices. Gomputing a basis as “good” as the private basis R, given 
only the non-reduced basis B, means approximating SBP. 

The message space is a “large enough” cube in Z”. A message m G Z” 
is encrypted into c = mB + e where e is an error vector uniformly chosen 
from {—a, cr}", where <t is a security parameter. A ciphertext c is decrypted as 
[cR~^~\RB~^ (note: this is Babai’s round method [7] to solve GVP). But an 
eavesdropper is left with the GVP-instance defined by c and B. The private 
basis R is generated in such a way that the decryption process succeeds with 
high probability. The larger ct is, the harder the GVP-instances are expected to 
be. But a must be small for the decryption process to succeed. 



Improvements. In the original scheme, the public matrix B is the multi- 
plication of the secret matrix by sufficiently many unimodular matrices. This 
means that without appropriate precaution, the public matrix can be as large 
as 0(n®logn) bits.^ Micciancio [83] therefore suggested to define instead B as 
the Hermite normal form (HNF) of R. Recall that the HNF of an integer square 
matrix R in row notation is the unique lower triangular matrix with coefficients 
in N such that: the rows span the same lattice as R, and any entry below the 
diagonal is strictly less than the diagonal entry in its column. Here, one can see 
that the HNF of R is O(n^logn) bits, which is much better but still big. When 
using the HNF, one should encode messages into the error vector e instead of a 
lattice point, because the HNF is unbalanced. The ciphertext is defined as the 
reduction of e modulo the HNF, and hence uses less than O(nlogn) bits. One 
can easily prove that the new scheme (which is now deterministic) cannot be 
less secure than the original GGH scheme (see [83]). 



Security. GGH has no proven worst-case/ average-case property, but it is much 
more efficient than AD. Specifically, for security parameter n, key-size and en- 
cryption time can be O(n^logn) for GGH (McEliece is slightly better though), 

® A different construction for R based on tensor product was proposed in [41], but 
seems to worsen the decryption process. 

^ Since the determinant has 0(n log n) bits, one can always make the matrix smaller 
than 0(n® log n) bits. 
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vs. at least O(n^) for AD. For RSA and El-Gamal systems, key size is 0{n) and 
computation time is O(n^). The authors of GGH argued that the increase in size 
of the keys was more than compensated by the decrease in computation time. 
To bring confidence in their scheme, they published on the Internet a series of 
five numerical challenges [47], in dimensions 200, 250, 300, 350 and 400. In each 
of these challenges, a public key and a ciphertext were given, and the challenge 
was to recover the plaintext. 

The GGH scheme is now considered broken, at least in its original form, 
due to an attack recently developed by Nguyen [90]. As an application, using 
small computing power and Shoup’s NTL library [107], Nguyen was able to solve 
all the GGH challenges, except the last one in dimension 400. But already in 
dimension 400, GGH is not very practical: in the 400-challenge, the public key 
takes 1.8 Mbytes without HNF or 124 Kbytes using the HNF.® 

Nguyen’s attack used two “qualitatively different” weaknesses of GGH. The 
first one is inherent to the GGH construction: the error vectors used in the 
encryption process are always much shorter® than lattice vectors. This makes 
GVP-instances arising from GGH easier than general GVP-instances. The second 
weakness is the particular form of the error vectors in the encryption process. 
Recall that c = mB + e where e G {±(t}”. The form of e was apparently 
chosen to maximize the Euclidean norm under requirements on the infinity norm. 
However, by looking at that equation modulo some well-chosen integer (such as 
a or even better, 2a), it is possible to derive information on the message m, 
which in turn leads to a simplification of the original closest vector problem, by 
shortening the error vector e. The simplified closest vector problem happens to 
be within reach (in practice) of current lattice reduction algorithms, thanks to 
the embedding strategy that heuristically reduces GVP to SVP. We refer to [90] 
for more information. 

It is easy to fix the second weakness by selecting the entries of the error 
vector e at random in [—a - ■ ■ + a] instead of {±cr}. However, one can argue 
that the resulting GGH system would still be impractical, even using [83]. In- 
deed, Nguyen’s experiments [90] showed that SVP could be solved in practice 
up to dimensions as high as 350, for (certain) lattices with gap as small as 10. 
To be competitive, the new GGH system would require the hardness (in lower 
dimensions due to the size of the public key, even using [83]) of SVP for certain 
lattices of only slightly smaller gap, which means a rather smaller improvement 
in terms of reduction. Note also that those experiments do not support the prac- 
tical hardness of Ajtai’s variant of SVP in which the gap is polynomial in the 
lattice dimension. Besides, it is not clear how to make decryption efficient with- 
out a huge secret key (Babai’s rounding requires the storage of R~^ or a good 
approximation, which could be in [49] over 1 Mbytes in dimension 400). 



® The challenges do not use the HNF, as they were proposed before [83]. Note that 
124 Kbytes is about twice as large as McEliece for the recommended parameters. 

® In all GGH-like constructions known, the error vector is always at least twice as 
short. The situation is even worse in [41]. 
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4.3 The NTRU Cryptosystem 

Description. The NTRU cryptosystem [56], proposed by Hoffstein, Pipher and 
Silverman, works in the ring R = 'L[X\/{X^ — 1). An element F G R is seen as 
a polynomial or a row vector: F = ~ Fi, ■ ■ ■, To select 

keys, one uses the set £{di,d 2 ) of polynomials F G R such that di coefficients 
are equal to 1, ^2 coeffients are equal to -1, and the rest are zero. There are two 
small coprime moduli p < q : a, possible choice is q = 128 and p = 3. There 
are also three integer parameters df,dg and d^ quite smaller than N (which is 
around a few hundreds). 

The private keys are / G £{df,df — 1) and g G £{dg, dg). With high proba- 
bility, / is invertible mod q. The public key ft- G i? is defined as ft = g/ f mod q. 
A message m G {—{p— l)/2 ■ ■ ■ + {p— l)/2}^ is encrypted into: e = {pcj)*h + m) 
mod q, where (j) is randomly chosen in £(d^,d^). The user can decrypt thanks 
to the congruence e * / = p4> * g + m* f (mod q), where the reduction is centered 
(one takes the smallest residue in absolute value). Since (j), /, g and m all have 
small coefficients and many zeroes (except possibly m), that congruence is likely 
to be a polynomial equality over Z. By further reducing e * / modulo p, one thus 
recovers m * f mod q, hence m. 

Security. The best attack known against NTRU is based on lattice reduction. 
The simplest lattice-based attack can be described as follows. Coppersmith and 
Shamir [33] noticed that the target vector f\\g G Z^^ (the symbol || denotes 
vector concatenation) belongs to the following natural lattice: 

Lcs = {F\\G G Z^^ I F = h* G mod q where F,Gg R}. 

It is not difficult to see that Lqs is a full-dimensional lattice in Z^^, with 
volume q^ . The volume suggests that the target vector is a shortest vector of 
Lcs (but with small gap), so that a SVP-oracle should heuristically output the 
private keys / and g. However, based on numerous experiments with Shoup’s 
NTL library [107], the authors of NTRU claimed in [56] that all such attacks 
are exponential in N, so that even reasonable choices of N ensure sufficient 
security. Note that the keysize of NTRU is only 0{Nlogq), which makes NTRU 
the leading candidate among knapsack-based and lattice-based cryptosystems, 
and allows high lattice dimensions. It seems that better attacks or better lattice 
reduction algorithms are required in order to break NTRU. To date, none of the 
numerical challenges proposed in [56] has been solved. However, cryptographic 
concerns have been expressed about the lack of security proofs for NTRU: there is 
no known result proving that NTRU or variants of its encryption scheme satisfy 
standard security requirements (such as semantic security or non-malleability,^*^ 
see [79]), assuming the hardness of a sufficiently precise problem. Besides, there 
exist simple chosen ciphertext attacks [60] that can recover the secret key, so 
that appropriate padding is necessary. 

NTRU without padding cannot be semantically secure since e(l) = m{l) (modg) as 
polynomials. And it is easily malleable using multiplications by X of polynomials 
(circular shifts). 
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5 The Hidden Number Problem 

5.1 Hardness of DifRe Heilman Bits 

There is only one example known in which the LLL algorithm plays a positive 
role in cryptology. In [18], Boneh and Venkatesan used LLL to solve the hidden 
number problem, which enables to prove the hardness of the most significant bits 
of secret keys in Diffie-Hellman and related schemes in prime fields. Recall the 
Diffie-Hellman key exchange protocol [36]: Alice and Bob fix a finite cyclic G 
and a generator g. They respectively pick random a,b € [1, jGj] and exchange 
and g^. The secret key is Proving the security of the protocol under “rea- 
sonable” assumptions has been a challenging problem in cryptography (see [12]). 
Computing the most significant bits of is as hard as computing itself, in 
the case of prime fields: 

Theorem 2 (Boneh- Venkatesan). Let q be an n-bit prime and g be a gener- 
ator ofZ*. Let e > 0 be fixed, and set I = i{n) = \e^/n]. Suppose there exists 
an expected polynomial time (in n) algorithm A, that on input q, g, g°“ and g^, 
outputs the i most significant bits of g°“^ . Then there is also an expected poly- 
nomial time algorithm that on input q, g, g°“ , g^ and the factorization of q — 1, 
computes all of g°“^ . 

The above result is slightly different from [18]. The same result holds for the 
least significant bits. For a more general statement when g is not necessarily a 
generator, and the factorization of g — 1 is unknown, see [51]. No such results are 
known for other groups (there is some kind of analogous result [113] for finite 
fields though). 

The proof goes as follows. We are given some g°“ and g^ , and want to compute 
gdb ^ We repeatedly pick a random r until is a generator of Z* (thanks to the 
factorization of g — 1). For each r, the probability of success is (j){q — l )/((7 — 1) > 
1/ log log g. Next, we apply A to the points and for many random 
values of t, so that we learn the most significant bits of gG+'^)b g{a-er)t ^ where 
g(a-i-r)t jg ^ random element of Z* since is a generator. Note that one can 
easily recover g°'^ from a = The problem becomes the hidden number 

problem (HNP): given t\, . . . ,td chosen uniformly and independently at random 
in Z*, and MSB^(ati mod q) for all i, recover a G Z^. Here, MSB^(a;) for x G Zq 
denotes any integer z satisfying [x — z] < q/2^. 

To achieve the proof, Boneh and Venkatesan presented a simple solution 
to HNP when £ is not too small, by reducing HNP to a lattice closest vector 
problem. We sketch this solution in the next section. One can try to prove 
the hardness of Diffie-Hellman bits for different groups with the same method. 
Curiously, for the important case of elliptic curve groups, no efficient solution 
is known for the corresponding hidden number problem, except when one uses 
projective coordinates to represent elliptic curve points. 

Due to an error in the proof of [18] spotted by [51]. 
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5.2 Solving the Hidden Number Problem by Lattice Reduction 

Consider an HNP-instance: let ti, ... ,td be chosen uniformly and independently 
at random in Z*, and Oi = MSB^(o:ti mod q) where a S Z^ is hidden. Clearly, the 
vector t = {tia mod q, . . .tdOi mod g, a/2^) belongs to the {d + l)-dimensional 
lattice L = L{q, £,ti, . . . ,td) spanned by the rows of the following matrix: 

/g 0 ••• 0 0 \ 

0 q -■■ ■. ■. 

: 0 : 

0 ... 0 g 0 

V^i td 1/27 

The vector a = (oi, . . . , a^, 0) is very close to L, because it is very close to t. 
Indeed, ||t — a|| < q^/d+ 1/2^. It is not difficult to show that any lattice point 
sufficiently close to a discloses the hidden number a (see [18, Theorem 5] or [98]): 

Lemma 3 (Uniqueness). Set d = 2 [ Vlog q \ and ^ = ^-^/logg + S. Let a be in 
Z*. Choose integers ti, . . . ,td uniformly and independently at random in Z*. Let 
a = (oi, . . . , Od, 0) he such that \{aU mod q) — Oi\ < qj2^ . Then with probability 
at least 5, all u G L with ||u — a|| < ^ are of the form: 

u = (fiP mod q, . . .tdjS mod q,l3l2^) where a = (3 (mod q). 

Since a is close enough to L, Babai’s nearest plane CVP approximation algo- 
rithm [7] yields a lattice point sufficiently close to a, which leads to: 

Theorem 4 (Boneh-Venkatesan). Let a be in Z*. Let O he a function defined 
by 0{t) = MSB^(at mod q) with t = [ Vlog q] + ["log log g] . There exists a deter- 
ministic polynomial time algorithm A which, on input t\, . . . ,td, 0{t \), . . . , 0{td) 
outputs a with probability at least 1/2 over t\,...,td chosen uniformly and in- 
dependently at random from Z*, where d = 2 [" Vlog q] . 

Thus, the hidden number problem can be solved using i = -y/log q -\- log log q bits. 
Using Schnorr’s improved lattice reduction algorithms, this can be asymptoti- 
cally improved to e-\/log q for any fixed £ > 0. One may also replace the bound ^ 
by 2^ and reduce the number of bits required by log log g. Then, the expected 

run time goes up by a factor -y/log q. One can alternately run \/log q copies of 
the algorithm in parallel. Theorem 2 is a simple consequence. 



5.3 Lattice Attacks on DSA 

Interestingly, the previous solution of the hidden number problem also has a dark 
side: it leads to a simple attack against the Digital Signature Algorithm [88,79] 
(DSA) in special settings (see [59,98]). Recall that the DSA uses a public element 
g G Zp of order q, a 160- bit prime dividing p— 1 where p is a large prime (at least 
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512 bits). The signer has a secret key a G Z* and a public key (} = g°‘ modp. 
The DSA signature of a message m is (r, s) G Z^ where r = mod p) mod q, 
s = + ar) mod q, h is SHA-1 hash function and fc is a random element 

in Z* chosen at each signature. 

It is well-known that the secret key a can easily be recovered if the random 
nonce k is disclosed, or if k is produced by a cryptographically weak pseudo- 
random generator such as Knuth’s linear congruential generator with known 
parameters [8]^^ and a few signatures are available. Recently, Howgrave-Graham 
and Smart [59] noticed that Babai’s nearest plane algorithm could heuristically 
recover a, provided that sufficiently many signatures and sufficiently many bits 
of the corresponding nonces k are known. This is not surprising, because the 
underlying problem is in fact very close to the hidden number problem. 

Indeed, assume that for d signatures (rj, Si) of messages rm, the £ least signif- 
icant bits of the random nonce ki are known to the attacker: one knows at < 2^ 
such that ki — at is of the form 2^bi. Then avi = Si{ai + 6^2^) — h{rrii) (mod q), 
which can be rewritten as: ari2~^s~^ = {ai — s~^h{mi)) -2~^ + bi (mod q). Let- 
ting ti = Ti2~^s~^ mod q, one sees that MSB^(o;ti mod q) is known. Recovering 
the secret key a. is therefore a slightly different hidden number problem in which 
the ti’s are not assumed to be independent and uniformly distributed over Z^, 
but are of the form ri2~^s~^ where the underlying kt’s are independent and 
uniformly distributed over Z*. In other words, HNP is an idealized version of 
the problem of breaking DSA (or related signature schemes) when the £ least 
significant bits (or more generally, £ consecutive bits) of the random nonce k are 
known for many signatures. It follows that Theorem 4 does not directly imply a 
provable attack on DSA in such settings. 

But an attacker can ignore the difference between the distribution of ri2~^s~^ 
and the uniform distribution, and simply identify the DSA problem to HNP. 
Since lattice reduction algorithms can behave much better than theoretically 
expected, one can even hope to solve CVP exactly, yielding better bounds to 
Theorem 4. It is straightforward to extend Theorem 4 to the case where a CVP- 
oracle is available, by going through the proof of Lemma 3. For the case of a 
160-bit prime q as in DSA, one obtains that HNP can be solved using respectively 
£ = 3 bits and d = 160, or ^ = 7 bits and c? = 85 respectively, when an 
oracle for CVPoo or CVP is available (see [98]). In fact, the bounds are even 
better in practice. It turns out that using standard lattice reduction algorithms 
implemented in Shoup’s NTL library [107], one can often solve HNP for a 160-bit 
prime q using £ = i bits and d = 100 (see [98]). 
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Note that even in the simple case where the parameters of the linear congruential 
generator are hidden, the attack of [8] does not apply. 
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6 Finding Small Roots of Low-Degree Polynomial 
Equations 

We survey an important application of lattice reduction found in 1996 by Cop- 
persmith [32], and its developments. These results illustrate the power of lin- 
earization combined with lattice reduction. 

6.1 Univariate Modular Equations 

The general problem of solving univariate polynomial equations modulo some 
integer N of unknown factorization seems to be hard. Indeed, notice that for 
some polynomials, it is equivalent to the knowledge of the factorization of N. 
And the particular case of extracting e-th roots modulo N is the problem of 
decrypting ciphertexts in the RSA cryptosystem, for an eavesdropper. Curiously, 
Coppersmith [32] showed using LLL that the special problem of finding small 
roots is easy: 

Theorem 5 (Coppersmith). Let P he a monic polynomial of degree S in one 
variable modulo an integer N of unknown factorization. Then one can find in 
time polynomial in ilogN,2^) all integers xq such that P{xq) = 0 (modN) and 

Related (but weaker) results appeared in the eighties [54,110].^^ We sketch a 
proof of Theorem 5, as presented by Howgrave-Graham [57], who simplified 
Coppersmith’s original proof (see also [62]). Coppersmith’s method reduces the 
problem of finding small modular roots to the (easy) problem of solving poly- 
nomial equations over Z. More precisely, it applies lattice reduction to find an 
integral polynomial equation satisfied by all small modular roots of P. The in- 
tuition is to linearize all the equations of the form x'^P{xy = 0 (modW) for 
appropriate integral values of i and j. Such equations are satisfied by any so- 
lution of P{x) = 0 (mod TV). Small solutions xq give rise to unusually short 
solutions to the resulting linear system. To transform modular equations into 
integer equations, the following elementary lemma^^ is used, with the notation 
lk(3^)ll = a/X) for any polynomial r(x) = J^UiX^ G Z[x]: 

Lemma 6. Let r(x) G Z[x] he a polynomial of degree n and let X be a positive 
integer. Suppose ]|r(a;A)j| < Lfr{xo) = 0 {modN’^) with ja^oj < X, then 

r(xo) = 0 holds over the integers. 

Now the trick is to, given a parameter h, consider the n = (h+ 1)<5 polynomials 
qu,v{x) = N^~'"x'^P{xy , where 0 < u < <5 — 1 and 0 < v < h. Notice that any 
root xo of P(x) modulo is a root modulo of qu,v(x), and therefore, of 
any integer linear combination r(x) of the ( 7 „,„(a;)’s. If such a combination r(x) 

Hastad [54] presented his result in terms of system of low-degree modular equations, 
but he actually studies the same problem, and his approach achieves the weaker 
bound 

A similar lemma is used in [54]: the bound eventually obtained in [54] is weaker 
because only = 1 is considered. Note also the resemblance with [73, Prop. 2.7]. 
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further satisfies ||r(a;X)|| < then by Lemma 6, solving the equation 

r{x) = 0 over Z yields all roots of P{x) modulo N less than X in absolute 
value. This suggests to look for a short vector in the lattice corresponding to the 
qu,v{xX)’s. More precisely, define the n x n matrix M whose z-th row consists 
of the coefficients of qu,v{xX), starting by the low-degree terms, where v = 
[(z — 1) /5\ and u= (z — 1) — 5v. Notice that M is lower triangular, and a simple 
calculation leads to det(M) = We apply an LLL-reduction to 

the full-dimensional lattice spanned by the rows of M. The first vector of the 
reduced basis corresponds to a polynomial of the form r(xX), and has Euclidean 
norm ||r(a;X)|j. The theoretical bounds of the LLL algorithm ensure that: 

||r(xX)|| < 2("-i)/^det(M)i/" = 

Recall that we need ||r(a;X)|| < / y/n to apply the lemma. Hence, for a given 

h, the method is guaranteed to find modular roots up to X if: 

" V2 

The limit of the upper bound, when h grows to oo, is Theorem 5 

follows from an appropriate choice of h. This result is practical (see [35,58] for 
experimental results) and has many applications. It can be used to attack RSA 
encryption when a very low public exponent is used (see [13] for a survey). Boneh 
et al. [17] applied it to factor efficiently numbers of the form N = p'~q for large r. 
Boneh [14] used a variant to find smooth numbers in short interval. See also [10] 
for an application to Chinese remaindering in the presence of noise. 



Remarks. Theorem 5 is trivial if P is monic. Note also that one cannot hope to 
improve the (natural) bound for all polynomials and all moduli N. Indeed, 
for the polynomial P{x) = x^ and N = where p is prime, the roots of P mod N 
are the multiples of p. Thus, one cannot hope to find all the small roots (slightly) 
beyond = p, because there are too many of them. This suggests that even a 
SVP-oracle (instead of LLL) should not help Theorem 5 in general, as evidenced 
by the value of the lattice volume (the fudge factor yielded by LLL is 

negligible compared to det(M)^/”). It was recently noticed in [10] that if one 
only looks for the smallest root mod N, an SVP-oracle can improve the bound 
7V1/<5 for very particular moduli (namely, squarefree N of known factorization, 
without too small factors). Note that in such cases, finding modular roots can 
still be difficult, because the number of modular roots can be exponential in the 
number of prime factors of N. 

6.2 Multivariate Modular Equations 

Interestingly, Theorem 5 can heuristically extend to multivariate polynomial 
modular equations. Assume for instance that one would like to find all small 
roots of P{x,y) = 0 (modA^), where P{x,y) has total degree S and has at 
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least one monic monomial of maximal total degree. If one could obtain 

two algebraically independent integral polynomial equations satisfied by all suf- 
ficiently small modular roots (x,y), then one could compute (by resultant) a 
univariate integral polynomial equation satisfied by x, and hence find efficiently 
all small (x,y). To find such equations, one can use an analogue of lemma 6 
to bivariate polynomials, with the (natural) notation ||r(a;,y)|| = ^or 

r{x, y) = aijxY : 

Lemma 7. Let r(x, y) G Z[a;, y] he a sum of at most w monomials. Assume 
\\r{xX,yY)\\ < for some X,Y > 0. If r{xo,yo) = 0 (modN^) with 

|a:o| < X and |j/o| < Y, then r(xo,yo) = 0 holds over the integers. 

By analogy, one chooses a parameter h and select r(x, y) as a linear combination 
of the polynomials qui,u 2 ,v{x, y) = y“^P(a;, yY , where ui + u^ + dv < h6 

and u\,U 2 ,v > 0 with ui < a or U 2 < <5 — a. Such polynomials have total degree 
less than h5, and therefore are linear combinations of the n= {h5 +\){h5 + 2) /2 
monic monomials of total degree < Sh. Due to the condition ui < a or U 2 < S—a, 
such polynomials are in bijective correspondence with the n monic monomials 
(associate to qui,u 2 ,v{x,y) the monomial a;“i+«'“y“2+«(<5-a)^^ Qjjg represent 
the polynomials as n-dimensional vectors in such a way that the n x n matrix 
consisting of the quj,u 2 ,v{xX,yYys (for some ordering) is lower triangular with 
coefficients on the diagonal. 

Now consider the first two vectors ri{xX,yY) and r 2 {xX,yY) of an LLL- 
reduced basis of the lattice spanned by the rows of that matrix. Since any root 
(xo,yo) of P{x,y) modulo fV is a root of qui,u 2 ,v{x,y) modulo N^, we need 
\\ri{xX,yY)\\ and ||r 2 (a;Al, yy)|| to be less than to apply Lemma 7. A 

(tedious) computation of the triangular matrix determinant enables to prove that 
ri{x, y) and V 2 {x^ y) satisfy that bound when XY < and h is sufficiently 

large (see [62]). Thus, one obtains two integer polynomial bivariate equations 
satisfied by all small modular roots of P{x,y). 

The problem is that, although such polynomial equations are linearly inde- 
pendent as vectors, they might be algebraically dependent, making the method 
heuristic. This heuristic assumption is unusual: many lattice-based attacks are 
heuristic in the sense that they require traditional lattice reduction algorithms 
to behave as SVP-oracles. An important open problem is to find sufficient con- 
ditions to make Coppersmith’s method provable for bivariate (or multivariate) 
equations. Note that the method cannot work all the time. For instance, the 
polynomial x — y has clearly too many roots over and hence too many roots 
mod any N (see [32] for more general counterexamples). 

Such a result may enable to prove several attacks which are for now, only 
heuristic. Indeed, there are applications to the security of the RSA encryption 
scheme when a very low public exponent or a low private exponent is used 
(see [13] for a survey), and related schemes such as the KMOV cryptosystem 
(see [9]). In particular, the experimental evidence of [15,9] shows that the method 
is very effective in practice for certain polynomials. 
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Remarks. In the case of univariate polynomials, there was basically no choice 
over the polynomials qu,v{x) = used to generate the appropri- 

ate univariate integer polynomial equation satisfied by all small modular roots. 
There is much more freedom with bivariate modular equations. Indeed, in the 
description above, we selected the indices of the polynomials qui,u 2 ,v(x,y) in 
such a way that they corresponded to all the monomials of total degree < h6, 
which form a triangle in when a monomial x'‘y^ is represented by the point 
(z, j). This corresponds to the general case where a polynomial may have several 
monomials of maximal total degree. However, depending on the shape of the 
polynomial P{x, y) and the bounds X and Y , other regions of (ui, U 2 , v) might 
lead to better bounds. 

Assume for instance P{x, y) is of the form x^^y^'« plus a linear combination 
of x'^y^’s where i < Sx, j < Sy and i + j < Sx + Sy. Intuitively, it is better 
to select the (ui,U 2 ,u)’s to cover the rectangle of sides hSx and h 6 y instead of 
the previous triangle, by picking all qu-i,u 2 ,v{x, y) such that u\ + vSx < hSx and 
U2 + vSy < hSy, with ui < Sx or U2 < Sy. One can show that the polynomials 
ri(x, y) and r 2 {x, y) obtained from the first two vectors of an LLL-reduced basis 
of the appropriate lattice satisfy Lemma 7, provided that h is sufficiently large, 
and the bounds satisfy Boneh and Durfee [15] applied similar 

and other tricks to a polynomial of the form P(x,y) = xy + ax + b. This allowed 
better bounds than the generic bound, leading to improved attacks on RSA with 
low secret exponent. 

6.3 Multivariate Integer Equations 

The general problem of solving multivariate polynomial equations over Z is also 
hard, as integer factorization is a special case. Coppersmith [32] showed that 
a similar^® lattice-based approach can be used to find small roots of bivariate 
polynomial equations over Z: 

Theorem 8 (Coppersmith). Let P{x, y) he a polynomial in two variables over 
Z, of maximum degree S in each variable separately, and assume the coefficients 
of f are relatively prime as a set. Let X, Y be hounds on the desired solutions 
xq, yo. Define P{x, y) = P{Xx, Yy) and let D he the absolute value of the largest 
coefficient of P. If XY < £)2/(3(5)^ then in time polynomial in {logD, 2^), we can 
find all integer pairs {xo,yo) such that P{xo,yo) = 0, |a;o| < X and |j/o| < Y. 

Again, the method extends heuristically to more than two variables, and there 
can be improved bounds depending on the shape^® of the polynomial (see [32]). 
Theorem 8 was introduced to factor in polynomial time an RSA-modulus^^ 
N = pq provided that half of the (either least or most significant) bits of either 

However current proofs are somehow more technical than for Theorem 5. A simpli- 
fication analogue to what has been obtained for Theorem 5 would be useful. 

The coefficient 2/3 is natural from the remarks at the end of the previous section for 
the bivariate modular case. If we had assumed P to have total degree 5, the bound 
would be XY < 

p and q are assumed to have similar size. 
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p or q are known (see [32,14,16]). This was sufficient to break an ID-based RSA 
encryption scheme proposed by Vanstone and Zuccherato [111]. Boneh et al. [16] 
provide another application, for recovering the RSA secret key when a large 
fraction of the bits of the secret exponent is known. Curiously, none of the 
applications cited above happen to be “true” applications of Theorem 8. It was 
later realized in [58,17] that those results could alternatively be obtained from 
a (simple) variant of the univariate modular case (Theorem 5) . 



7 Lattices and RSA 

Section 6 suggests to clarify the links existing between lattice reduction and 
RSA [100], the most famous public-key cryptosystem. We refer to [79] for an 
exposition of RSA, and to [13] for a survey of attacks on RSA encryption. Recall 
that in RSA, one selects two prime numbers p and q of approximately the same 
size. The number N = pq is public. One selects an integer d coprime with 
(p{N) = [p — l)(q — 1). The integer d is the private key, and is called the RSA 
secret exponent. The public exponent is the inverse e of d modulo 4>{N). 

7.1 Lattice Attacks on RSA Encryption 

Small Public Exponent. When the public exponent e is very small, such 
as 3, one can apply Coppersmith’s method (seen in the previous section) for 
univariate polynomials in various settings (see [13,32,35] for exact statements): 

— An attacker can recover the plaintext of a given ciphertext, provided a large 
part of the plaintext is known. 

— If a message is randomized before encryption, by simply padding random 
bits at a known place, an attacker can recover the message provided the 
amount of randomness is small. 

— Hastad [54] attacks can be improved. An attacker can recover a message 
broadcasted (by RSA encryption and known affine transformation) to suf- 
ficiently many participants, each holding a different modulus N. This pre- 
cisely happens if one sends a similar message with different known headers 
or time-stamps which are part of the encryption block. 

None of the attacks recover the secret exponent d: they can only recover the 
plaintext. The attacks do not work if appropriate padding is used (see current 
standards and [79]), or if the public exponent is not too small. For instance, the 
popular choice e = 65537 is not threatened by these attacks. 



Small Private Exponent. When d < an old result of Wiener [114] 

shows that one can easily recover the secret exponent d (and thus the factor- 
ization of N) from the continued fractions algorithm. Boneh and Durfee [15] 
recently improved the bound to d < by applying Coppersmith’s tech- 

nique to bivariate modular polynomials and improving the generic bound. Note 
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that the attack is heuristic (see Section 6), but experiments showed that it works 
well in practice (no counterexample has ever been found). All those attacks on 
RSA with small private exponent also hold against the RSA signature scheme. A 
related result (using Coppersmith’s technique for either bivariate integer or uni- 
variate modular polynomials) is an attack [16] to recover d when a large portion 
of the bits of d is known (see [13]). 

7.2 Lattice Attacks on RSA Signature 

The RSA cryptosystem is often used as a digital signature scheme. To prevent 
various attacks, one must apply a preprocessing scheme to the message, prior to 
signature. The recommended solution is to use hash functions and appropriate 
padding (see current standards and [79]). However, several alternative simple 
solutions not involving hashing have been proposed, and sometimes accepted as 
standards. Today, all such solutions have been broken (see [45]), some of them 
by lattice reduction techniques (see [86,45]). Those lattice attacks are heuristic 
but work well in practice. They apply lattice reduction algorithms to find small 
solutions to (affine) linear systems, which leads to signature forgeries for certain 
proposed RSA signature schemes. Finding such small solutions is seen as a closest 
vector problem for some norm. 

7.3 Factoring and Lattice Reduction 

In the general case, the best attack against RSA encryption or signature is 
integer factorization. Note that to prove (or disprove) the equivalence between 
integer factorization and breaking RSA encryption remains an important open 
problem in cryptology (latest results [19] suggest that breaking RSA encryption 
may actually be easier). We already pointed out that in some special cases, 
lattice reduction leads to efficient factorization: when the factors are partially 
known [32], or when the number to factor has the form q with large r [17]. 

Schnorr [103] was the first to establish a link between integer factorization 
and lattice reduction, which was later extended by Adleman [2]. Schnorr [103] 
proposed a heuristic method to factor general numbers, using lattice reduction 
to approximate the closest vector problem in the infinity or the Li norm. Adle- 
man [2] showed how to use the Euclidean norm instead, which is more suited 
to current lattice reduction algorithms. Those methods use the same underlying 
ideas as sieving algorithms (see [30]): to factor a number n, they try to find 
many congruences of smooth numbers to produce random square congruences 
of the form (modn), after a linear algebra step. Heuristic assumptions 

are needed to ensure the existence of appropriate congruences. The problem of 
finding such congruences is seen as a closest vector problem. Still, it should be 
noted that those methods are theoretical, since they are not adapted to currently 
known lattice reduction algorithms. To be useful, they would require very good 
lattice reduction for lattices of dimension over at least several thousands. 

We close this review by mentioning that current versions of the Number Field 
Sieve (NFS) (see [72,30]), the best algorithm known for factoring large integers. 
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use lattice reduction. Indeed, LLL plays a crucial role in the last stage of NFS 
where one has to compute an algebraic square root of a huge algebraic number 
given as a product of hundreds of thousands of small ones. The best algorithm 
known to solve this problem is due to Montgomery (see [87,89]). It has been used 
in all recent large factorizations, notably the record factorization [28] of a 512- 
bit RSA-number of 155 decimal digits proposed in the RSA challenges. There, 
LLL is applied many times in low dimension (less than 10) to find nice algebraic 
integers in integral ideals. But the overall running time of NFS is dominated by 
other stages, such as sieving and linear algebra. 

8 Conclusions 

Lovasz’s algorithm and other lattice basis reduction algorithms have proved in- 
valuable in cryptology. They have become the most popular tool in public-key 
cryptanalysis. In particular, they play a crucial role in several attacks against 
the RSA cryptosystem. The past few years have seen new, sometimes provable, 
lattice-based methods for solving problems which were a priori not linear, and 
this definitely opens new fields of applications. Paradoxically, at the same time, 
a series of complexity results on lattice reduction has emerged, giving rise to an- 
other family of cryptographic schemes based on the hardness of lattice problems. 
The resulting cryptosystems have enjoyed different fates, but it is probably too 
early to tell whether or not secure and practical cryptography can be built using 
hardness of lattice problems. Indeed, several questions on lattices remain open. 
In particular, we still do not know whether or not it is easy to approximate the 
shortest vector problem up to some polynomial factor, or to find the shortest 
vector when the lattice gap is larger than some polynomial in the dimension. 
Besides, only very few lattice basis reduction algorithms are known, and their 
behaviour (both complexity and output quality) is still not well understood. And 
so far, there has not been any massive computer experiment in lattice reduction 
comparable to what has been done for integer factorization or the elliptic curve 
discrete logarithm problem. Twenty years of lattice reduction yielded surprising 
applications in cryptology. We hope the next twenty years will prove as exciting. 
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Abstract. This paper proposes an algorithm which, given a basis of a 
subspace of the space of cuspforms of weight 2 for Io(A) which is in- 
variant for the action of the Hecke operators, tests whether the subspace 
corresponds to a quotient A of the Jacobian of the modular curve Xq{N) 
such that A is the Jacobian of a curve C. Moreover, equations for such a 
curve C are computed which make the quotient suitable for applications 
in cryptography. One advantage of using such quotients of modular Jaco- 
bians is that fast methods are known for finding their number of points 
over finite fields [6]. Our results extend ideas of M. Shimura [13] who 
used only the full modular Jacobian instead of abelian quotients of it. 



1 Cab Curve 

First, we define Cab curve following Miura[9j. Let C be an algebraic curve defined 
over a perfect field K with a place P of degree one. Take the ring L{ooP) of 
functions on C which are holomorphic away from P: 

L{^P) = {/ e K{C) I VQif) > 0 (vg ^ P)}. 

All of the pole numbers —vp{f) at P of / € L{ooP) become a monoid Mp: 

Mp = {-vp{f) I / e L(ooP)}. 

Take a minimum system A = {a\, 02 , ... , at\ (oi < 02 • • • < Ot) of generators 
of Mp as a monoid: 



Mp — NqUi + N 0 O 2 + ■ ■ ■ + — (^)- 

As Mp is co-finite in No, we have gcd(ai, . . .,at) = 1. For A = {ai , . . . , at}, 
define a function tfM on Nq as 

i 

<pA{ni,...,nt) = '^airii (n = (rii) G Nq). 
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Definition 1 (Cab Order). For m = (mi, . . and n = (n\, . . ,,nt) € Nq, 

define an order >a, as 

m n 'FA(m) > 'FA(n) 
or 

= FA(n),mi =m,. rui-i = rij-i, m, < Ui. 

Then, the order becomes a monomial order, called ‘Cab order of type A”. □ 
We need to define two sets: 

B(A) = {the least m G Nq w.r.t Cab order of type A with 'J/A{m) = a \ a G (A)}, 
V(A) = {/ G No \ B(A) \l = m + n,mGfCo\ B(A),n G Nq n = (0, 0, . . . , 0)}. 

Miura[9] showed 

Theorem 1. Let C he an algebraic curve defined over a perfect field K with a 
place P of degree one. Then, if 

Mp={A), A= {ai,...,at}, ai < ■ ■ ■ < at 

holds, the curve C has a nonsingular affine model in t dimensional affine space 
with the defining equations 

Fra = X'^ + aiX^ + Y. (mGV(A)). (1) 

n^B{A) a{ti)<'^ Aim) 

There, I is a unique I G B(A) satisfying FAim) = Fa( 1), and ai 0, a„ G K. 

The affine curve F^ = 0 (m G V(A)), obtained from A = {ai, . . . ,at} 
(gcd(oi , . . .,at) = I, ai < ■ ■ ■ < at), is called a “Cab curve of type A”. 



Example: C3,s,7 Curve 

<^ 3 , 5, 7 curve, that is Cab curve of type (3, 5, 7}, is a space curve defined by three 
equations of the form: 

= at)XZ -\- a\X^ -t- a^XX -t- a^Z -t- a4X‘^ -t- a^X aQX -t- ar 

XZ = boX‘^ + biX'^X + b^XZ + bs,X^ + biXX + b^Z + b^X^ 

+ hyX + hfiX + 69 

= cqX^X + ciX'^Z + + c^X'^X + C4XZ + + cqXX 

+ GyZ + C^X^ + CgT + CioAl + Cn- 



( 2 ) 




Construction of Secure Cab Curves Using Modular Curves 



115 



2 Security Condition 

A discrete log based cryptosystem using the jacobian of a curve C over a field F, 
will be less secure than a standard 1024 bit RSA system, unless four conditions 
are satisfied. 

1. (Against Pollard’s rho algorithm) 

The order h of the Jacobian Jq has a prime factor I of 160 or more bits[10]. 

2. (Against FR attack) 

The prime factor I does not divide — 1 for small k [5] . 

3. (Against Riick attack) 

I should be coprime with 9 [12]. 

4. (Against Gaudry’s variant) 

q has (40 + log2(84(g — 1)) + log2(m)) or more bits[7,3j. 

3 Construction of Secure Cab Curves 

For definitions of the congruence subgroup Fq^N), the modular curve Xq{N), 
and so on, see [4] or [11]. 



3.1 Number of Points of a Simple Factor of a Modular Curve 

Let N be a natural number. Let C{N) be a Q- vector space with basis P^(Z/NZ). 
Let B{N) be the subspace of C{N) spanned by all elements of the form 

{c: d) + {—d : c), 

{c : d) + {c + d : —c) + {d : —c — d). 

Let C'o(N) be a Q-vector space spanned by T'o(A^)-cusps. Define the boundary 
map (5 : C{N) Co{N) by 



6{{c : d)) = [a/c] - [b/d] 

where integers a and b are chosen so that ad — be = 1, and set Z{N) = ker(<5). 
Note that B{N) C Z{N). Finally, define F[{N) = Z{N) / B{N), which is a Q- 
vector space of dimension 2g, where g is the genus of the modular curve Xo{N). 

Let F[^{N) be a +1 proper space of the star operator * : {c : d) 1 -^ (— c : d) 
on F[{N). 

Proposition 1 ([4]). Let T be the Hecke algebra of level N. As T -modules, 

H+{N) (8)qC~ 52(N). 



□ 
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Hecke operator Tp can be dealt with as an operator on For a prime p 

not dividing N, the operator Tp on H'^{N) is calculated by Heilbronn matrices 



R 



p. 



Tp{{c -.d))= ^ (c : d)M. 

MGRp 

An algorithm for computing Heilbronn matrices is given in page 22 of [4] . 

A simple factor A of the Jacobian Jq{N) corresponds to a simple T-submodule 
K of one-to-one. The dimension of A as an abelian variety is equal to 

the dimension of A' as a vector space over Q. Using Eichler-Shimura relation [11] 
one finds the formula for the number of points on a factor A over a prime field 
Fp for a prime p not dividing N, 

'iA/¥p = net{x'^ -T p\k x+p) \x=i ■ (3) 



Example: Level 


97. 


Let N = 


97. H+{97) 


is 


7-dimensional over Q 


with 


a basis 


{51)52, 


• •,57}: 


























5i = 


(44 


1 )- 


(88 


1 ) + 


(91 


:1), 


















52 = 


(70 


1 ) + 


(87 


1 )- 


(88 


:1) + 


(91 : 


1 ) 


- (92 


1 ) + 


(93 


1 )- 


(94 


1 ), 


53 = 


(78 


1 )- 


(92 


1 ) + 


(93 


:1)- 


(94 : 


1 ), 














54 = 


(79 


1 ) + 


(87 


1 )- 


(88 


:1) + 


(91 : 


1 ) 


- (92 


1 ) + 


(93 


1 )- 


(94 


1 ), 


55 = 


(83 


1 )- 


(90 


1 ), 






















56 = 


(89 


1 )- 


(91 


1 ), 






















57 = 


(95, 


1 ). 



























Calculating the characteristic polynomial of T 2 using the basis {gi, P2, ■ ■ ■ , 57} 
and factoring it over Q, we get 

(—1 -I- 3a; -I- 4a;^ -I- a;^)(— 1 -I- 6a; — a;^ — 3a;^ -I- a;^). (4) 



This leads to the guess that Jo{Q7) is factored to the product of 3-dimensional 
simple abelian variety A3 and 4-dimensional simple abelian variety A4. 

Let it's be the T-submodule of iJ+(97) corresponding to A3. K 3 is spanned 
by proper vectors of the irreducible factor — 1 -|- 3a; -I- 4a;^ -I- a;^ of Equation (4). 
Using this, we can find a basis {/i, /2, /a} of K 3 over Q: 

/i = 8 • (44 : 1) -h 30 • (70 : 1) - 16 • (78 : 1) -k 2 • (79 : 1) -k 20 • (83 : 1) 

-k 32 • (87 : 1) - 40 • (88 : 1) -k 35 • (89 : 1) - 20 • (90 : 1) -k 5 • (91 : 1) 

- 16 • (92 : 1) -k 16 • (93 : 1) - 16 • (94 : 1) -h 39 • (95 : 1), 

/2 = -6 • (44 : 1) - 68 • (70 : 1) -k 12 • (78 : 1) -h 44 • (79 : 1) - 15 • (83 : 1) 

- 24 • (87 : 1) -k 30 • (88 : 1) - 49 • (89 : 1) -k 15 • (90 : 1) -k 19 • (91 : 1) 

-h 12 • (92 : 1) - 12 • (93 : 1) -k 12 • (94 : 1) - 52 • (95 : 1), 
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/3 = 50 • (44 : 1) + 142 • (70 : 1) - 9 • (78 : 1) - 124 • (79 : 1) + 34 • (83 : 1) 

+ 18 • (87 : 1) - 68 • (88 : 1) + 105 • (89 : 1) - 34 • (90 : 1) - 37 • (91 : 1) 

- 9 • (92 : 1) + 9 • (93 : 1) - 9 • (94 : 1) + 130 • (95 : 1). 

If As happens to be a Jacobian variety Jc of some curve C, the basis {fi, f 2 , h} 
should give a basis of regular differential forms on the curve C. It turns out that 
this is indeed the case. 



16529, Tp 


is represented by the 


matrix 








/ -36 


68 


-1 


67/2 


-55 


53/2 


34 \ 




138 


120 


29 


9 


3 


20 


-50 




-136 


-305 


-185 


-321 


-85 


-322 


-162 




0 


0 


0 


114 


0 


17 


53 




-110 


85 


82 


93 


145 


95 


34 




0 


0 


0 


19 


0 


171 


-17 




V 0 


0 


0 


17 


0 


-2 


167 / 



with respect to the basis {51, 52, • • • j 57}- 

Using the Eichler-Shimura relation, one computes from this the characteristic 
polynomial /o of Frobenius <Tp at p (on C torsion or on a Tate module of ^3): 

/o = (a;® - 452x® + 115418x'‘ - 17978899®® + 1907744122®^ - 123489944132® 

+ 4515852403889) • (®® - 44®’’ + 31601®® - 1865601®® + 749060774®'’ - 
30836518929®® + 8633640983441®^ - 198697505771116® + 74642524383881281) 

So, the characteristic polynomial / of dp over Ks is the irreducible factor of /o 
of sixth degree: 

®® -452®® + 115418®'’ - 17978899®® + 1907744122®^ - 123489944132® + 4515852403889. 
The number ho of points of As over Fp is obtained by substituting ® = 1 for /: 

ho = /(I) = 4394252339947. 

Since the characteristic polynomial of the fifth power of Frobenius is also easily 
calculated from Tp, the number h of points over a degree five extension F® is 
obtained similarlily: 

h=4394252339947x427379515481622744216694600721926448140291414819361. 

It is immediately verified that q = and h in the above satisfies the security 
conditions 1,2,3, and 4. 

3.2 Defining Eqnation of a Simple Factor of a Modular Curve 

Using the method of [13], we determine whether or not a given simple factor of 
Jo(fV) is a Jacobian Jc of some algebraic curve, and if it is, we find a defining 
equation of the corresponding curve C . 

First, we give Shimura’s result for hyperelliptic modular curves. 
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Algorithm 1 (Defining Equation for Hyperelliptic Modular 
Curves [13]) 

Input: a basis {fi{z), f 2 {z), . . . , fg{z)} for 52 (A) of a hyperelliptic level N 
Output: a defining polynomial y^ — _ . . . _ u 2 g +2 

1° Calculate a Fourier expansion of every cusp form fi{z), and normalize them 
into the following manner: 

fi{z) = + • • • + + • • • 

f2{z) = + S2,gq^ hS2,g+i9®+*H 



fg(z) = g+Sg,29^H hSg,g+i9®+*H . 

We only need Fourier coefficients of at most (3g + 3)-th degree. 



( 5 ) 



/2 q dx 

fi fi dq 

3° Calculate coefficients oi, 02, • • • recursively as follows: 

y 2 _ 3,29+2 ^ 1 

yl _ ^2g+2 _ a^a;2g+l ^ 02?"^® + ' 



The principle of Algorithm 1 is as follows. When a modular curve Ao(A) is 
hyperelliptic, the cusp ooz G H is not a Weierstrass point. So, Xq{N) has an 
affine model with the cusp ooz as one of the two points at infinity: 

y 2 ^ 3;2g+2 _|_ a^a;2g+i _| 1 _ ^23+2. 



Let a be one of the roots of the right-hand side, then a basis of regular differential 
forms on Xq(N) is given by 

,dx {x — a)dx (a; — 



On the other hand, a basis of regular differential forms on Xq{N) is also given 

by 

{fidz,f 2 dz,...,fgdz}. 

Therefore, we can suppose 



h 

/i’ 



y = 



q dx 
fi dq' 



For a hyperelliptic curve C obtained as a factor of a modular curve, the cusp 
ooz G MI may be its Weierstrass point. (For example, the hyperelliptic curve of 
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genus 2 obtained as a factor of Xo(68).) But, also in this case, C has an affine 
model with the cusp ooz as a unique point at infinity: 



y -2 ^ 3,29+1 



a\x 



2 g+l 



- 02g+l- 



Putting one of the roots of the right-hand side as a, a basis of regular differential 
forms on C is given by 

,dx {x — a)dx {x — a)^~^dx. 



On the other hand, regular differential forms is also spanned by 



{fidz,f2dz,...,fgdz}, 

where {/i, • • • , fg} is a basis of T-submodule of S2{N) corresponding to C. There- 
fore, Normalizing {/i, • • • , /g} as 

fi{z) = 9^® ^ + si^g+iq^^ '^ + ■ ■ ■ 
f2{z) = g2g-3 + ^ . 



fg{z) — d + Sg,2 + ■ ■ ■ , 



we can also suppose 



/2 q dx 

^ = Tl~- 

h h dq 



Note when the cusp ooz is a Weierstrass point, x has a pole of order two at a 
point at infinity. 

Thus, we get 



Algorithm 2 (Modular Kyperelliptic; Cusp ooz as a Weierstrass Point) 

Input: a basis {fi{z), f2{z), , fg{z)} of a T-submodule of 52 (A) 

Output: a defining polynomial — aix^^ _ . . . _ 029+1 

1° Calculate a Fourier expansion of every cusp form fi{z), and normalize them 
into the following manner: 



fi{z) = 9^® ^ + si,g+iq'^^ ^ H 

f2{z) = g29-3 + ^^_^^29-4 ^ . 



fg{z) — q + 59,2 + • • • ■ 



o 



X 




q dx 
fi dq 



2 ' 
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3° Calculate coefficients ai, 02 , • • • recursively as follows: 



y 



+ • 



'/ - = 029""^®+^ + • • • 



In general, for an algebraic curve C which is not hyperelliptic, letting a basis 
of space of regular differential forms H^{Qq) be {oji, ■ ■ ■ , ojg}, the map 

^:C — > 

P - fl,^iP),...,^(P)) 

\ UJi UJi J 

is an embedding morphism, and its image Im(<?) is a nonsingular algebraic curve 
in PS“^, called a “canonical curve” of C. 

In the case of modular curve or its factor, its canonical curve is just an 
algebraic curve in defined by the relations among {fi, ■ ■ ■ , fg}, which is a 
basis of the corresponding T-submodule of 52 (-/V). 

A canonical curve of genus three is a plane quartic curve. As Shimura pointed 
out [13], the following Theorem 2 is useful for a canonical curve of genus four or 
more. 

Theorem 2 (Petri’s Theorem [1]). Let C be a canonical curve of genus four 
or more. Then C is an intersection of some quadratic hyper surf aces, or an in- 
tersection of some quadratic and cubic hyper surf aces. 

By Theorem 2, for a curve C obtained as a factor of a modular curve, we only 
need to find quadratic or cubic relations among fi, - ■ ■ , fg in order to obtain a 
canonical curve of C. Shimura estimates the number of relations as in Table 1 
[13]. 



Table 1. Number of equations for canonical curves 



genus 


equations 


3 


one quartic relation 


4 


one quadratic and one cubic relations 


5 


three quadratic relations 



Each explicit relation is obtained easily using the Fourier expansions of a 
basis {fi{z)j2{z),...,fg{z)}. 

Take an abelian variety A obtained as a simple factor of Jq{N). Let K he a 
T-submodule of S 2 {N) corresponding to A, and {/i, . . . , fg} be its basis over Q. 

Now, we can determine whether A is a Jacobian Jc of some algebraic curve 
or not, and if it is, we can find a defining equation of the corresponding curve 
C, as follows: 
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Algorithm 3 (Defining Equation of Simple Factor of a Modular 
Curve) 

Input: a basis {/i, . . . , fg} of a T-submodule of S 2 {N) over Q, corresponding to 
a simple factor A of Jo{N) 

Output: ‘null’ or a defining polynomial F of an algebraic curve C with Jacobian 
Jc^A 

1° Calculate a Fourier expansion of every cusp form fi{z), and determine the 
cusp ooz is a Weierstrass point or not. That the cusp ooz is not a Weier- 
strass point is equivalent to the fact that {fi, . . . , fg} are expanded just as in 
Equation (5) in Algorithm 1. 

2° Assume A is a Jacobian of some hyperelliptic curve. Calculate a defining 
polynomial F{x, y) of the hyperelliptic curve, using Algorithm 1 when the 
cusp ooz is not a Weierstrass point, or Algorithm 2 when the cusp ooz is a 
Weierstrass point. 

3° Check the validity of the polynomial F{x,y). That is, substitute x = ^, 
and y = for F{x,y), and see whether the resulting Fourier coefficients 
vanish. If it is, output F{x, y) and terminate. 

4° Assume C is a Jacobian of some non-hyperelliptic curve C , and calculate 
the canonical curve of C. That is, find all the quadratic or cubic relations F 
among {/i, . . . , fg}. And see whether the curve defined by F is nonsingular. 
If it is, output F and terminate. Else A is supposed to be not a Jacobian 
variety, and output ‘null’. 

Algorithm 3 is not strict with mathematics, of course. It is not proved that 
the output of Algorithm 3 defines an algebraic curve with Jacobian A. However, 
remember that our aim is to construct a secure Cat curve. It is sufficient that 
the resulting curve has a Jacobian of the expected order in fact. 



Example: Level 97. In section 3.1, we guessed that Jacobian Jo(97) has a 
three-dimensional simple factor A 3 . Also, we obtained the basis {/i,/ 2 ,/ 3 } of 
the corresponding T-submodule K3 of 52 (A). Here, we perform Algorithm 3 
with the basis {fi, f 2, f 3} as an input. 

1° Calculating and normalizing Fourier expansions of {/i, / 2 , fs}, we get 

fi ^ q - q - 2q - q + q + 4q - 2q + 3q +q - q -7q - q + q ••• 

f2^q^~ 3/ - g® - -t + q^ + 2q^° -t -t - q^^ - iq^* - + ••• 

h^q-Aq^- 5g® - - q’ + 9g® - q^ + Sq^° - -t 7q^^ - 2q^^ - 3g^^ . 

Coefficients are calculated up to 80-th degree. From this, we know the cusp 
ooz is not a Weierstrass point. 

2° Assuming A 3 is an Jacobian of some hyperelliptic curve, we calculate a defin- 
ing equation of the hyperelliptic curve, using Algorithm 1. 

1°° Fourier expansions of {/i, / 2 , /a} was already computed at 1°. 
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2°° We obtain 




-lio2 4, .5 oTiryS, 9 rlOi-ioll nl3, 

— 1 + ^ + 2(2 — Q 4^^ — 2^ “1“ 7^ “t ^ — 5(2 “t 13(2 — 9^ + • • • 

q dx 

^ h dq 

= -8 - (2~'‘ - q~^ - 3/(2^ - 2/g - 14(2 - 7(2^ - 28(2® - 57(2^‘ . (6) 

Actually, coefficients are calculated up to 75-th degree. 

3°° We obtain the defining polynomial 

- 23 -t 182x - 241®^ -t 210®® - 136®'‘ -t 62x® - 21®® -t 6®’’ - ®® -t (7) 

3° Substituting Equation (6) for Polynomial (7), we encounter 

70(2 + 14(2^ - 300(2® + S98q'^ -t 174(2® - 1106(2® + 930(2’’ + 479(2® - 472(2® - 1572(2®° . 

As coefficients does not vanish, we determine A3 is not a Jacobian of any 
hyperelliptic curve. 

4° Assuming A3 is an Jacobian of some non- hyperelliptic curve C, we calculate 
the canonical curve of C. As the genus of C is three, the defining polynomial 
is a single quartic equation F among Z = fi,Y = fi,X = /3 . Using the 
above Fourier expansion of A, Y, Z, we obtain the unique relation 

F = -2A'®-A®y-3A®y®+6A®Z+3A®yZ+Ay®Z-ty®Z-5A®Z®-y®Z®-tAZ®. 

As y = 0 defines a nonsingular curve, we determine the simple factor A3 is 
a Jacobian of the curve F = 0. 



3.3 Cab Model of a Simple Factor of a Modular Curve 

In the last section, we got an explicit defining equation of a simple factor of a 
modular curve. Here we translate it into a Cab curve. 

In the hyperelliptic case, we obtain the equation of the form 

y2 ^ ^2g+2 ^ 02g+2, 

besides Cab curve of type {2, 2g + 1}. Factoring the right-hand side, we get 

= {x + Ai)(® -l- A2) • • • (® + A2g+2), 

and dividing two sides by (® -I- A)^®4-2^ gg^ 

2 2^+2 

y = 1 . fr (1 -p 

(® + A)29+2 U ^ ^ a; + Ai ^ 

So, putting 

V _ ^ Y = y. 

® + Ai’ (® + Ai) 9 +i’ 
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we get 

2g+2 

= n 

i=2 

This is a Cab curve of type {2, 2g + 1}. 

We now consider the non-hyperelliptic case. For a non-hyperelliptic simple 
factor, we obtained its canonical curve C . By Theorem 1, in order to translate 
C into a Cab curve, we only need to find the generator A = (oi, 02 , • • • , at) of 
the monoid Mq and to find the function ft G L(ooQ) (i = 1,2, ■ ■ ■ ,t) with pole 
order i at Q , for some rational point Q on C. Therefore, all we have to do is to 
find a basis L{mQ) for some rational point Q on C (3 < m < at). 

As the canonical curve C is nonsingular, it is not difficult to find a basis of 
L{D) for any divisor D [8]: 

Proposition 2. Let D — ^ riiPi — Y^ '^jQj ® divisor on a nonsingular curve 
C with non-negative integers ni,mj. Let L\ = f]Lp),l 2 = F/q^, where Lp is the 
maximal ideal corresponding to P of the coordinate ring R of C. Fix (0 yf)V/ G 

h- 

Then, we have 

L{D) = {^^\gGfh-.h}. 

Proof. As C is nonsingular, any element in L(D) can be written as j for some 
g G R. Then, 

-j G L{D) 

^ jh C I 2 

g & fh ■ h 

□ 



The number of equations for a Cab curve becomes the smallest when we take 
a Weierstrass point as the base point. So, it is desirable to choose a Weierstrass 
point as the point Q in the above. When the genus is three, Weierstrass points 
of a canonical curve are easily found. 

Let C be a canonical curve of genus three. Let AT be a canonical series of C. 
As dim(A') = 2, that a point Q is Ei Weierstrass point is equivalent to the fact 
that the tangent line at the point Q meets the curve C with the multiplicity 
three or more. So, a Weierstrass point Q is a common zero of 

f{x,y) = 0 
Da,bf{x,y) = 0 
Dalfi.x,y) = 0 , 

where 



,( 2 ) 



Da,bf{x, y) = adfjdx + hdfjdy. 
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Example: Level 97. We saw that the Jacobian Jo(97) of the modular curve 
Jfo(97) has a simple factor A3 of dimension three, and that A3 is a Jacobian 
of an algebraic curve C{rei. section 3.2). The defining equation of the canonical 
curve of C was given by 

/ = —2x'^ — x^y — ix^y^ + 6x^ + 3x^y + xy^ + y^ — 5x^ — y^ + x. 

For p = 16529, we find a Weierstrass point of the curve / = 0 over Fp. As 
equations 

f{x, y) = X — 5x^ + — 2x^ + 3x^y — x^y — y^ + xy^ — 3x^y^ + y^ 

Da,bf{x, y) = a — IQax + 3x^ + ISaa;^ — x^ — 8ax^ — 2y + 2xy + 6axy — 6x^y 
— 3ax^y + 3y^ + ay^ — Qaxy^ 

D^^\f{x, y) = —2 — lOa^ + 2x+ 12ax + 36a^x — &x^ — dax^ — 24a^x^ + 6y 
+ 4ay + 6a^y — 24axy — da^xy — da^y^ 

has a common zero 



a = 12900, X = 13695, y = 14705, 



Q = (13695, 14705) is a Weierstrass point. 

Calculating l{m) = dim L{mQ) (m = 3, 4, 5, 6, 7) by Prop. 2, we have 

;(3) = 2, l{4) = 2, ;(5) = 3, l{d) = 4, l{7) = 5. 

So, we know gap sequences at the point Q is 1,2,4. Hence, 

Mq = (3,5,7). 

Thus, the curve C is a <73,5,7 curve. 

The function X G L{3Q) with vq{X) = —3, the function Y G L{4Q) with 
vq{Y) = —4, and the function Z G L{5Q) with vq{Z) = —5 are given by 

X = (12855 + 11167a; + 5996®^ + + 9720y + 10529x1/ + 4636x^1/ + 10496//^ 

+ 10744x//^)/(13280 + 13941// + 5472//^ + y^) 

Y = (8608 + 6182x + 8423x^ + 15577x® + 13719// + 7604x// + 424x^// + 8263x®// 

+ 7442//^ + 9157x//^ + 7894x^//^ + 4131x®//^ + 14194//® + 12726x//® 

+ 9702x^//® + 15348//"^ + 5202x//'‘)/(9403 + 5617// + 568//^ + 13412//® 

+ 9120/ +//®) 

Z = (10644 + 13291X + 7571x^ + 8617x® + 2836y + 15714x// + 1350x^y + x®// 

+ 667//^ + 3987x//^ + 11840x^y^ + 2036x®y^ + 1947//® + 1150xy® 

+ 12002x^//® + 6207x®//® + 15337/ + 7047x/ + 8184x^/ + 13431x®//'‘ 

+ 8564//® + 5258x//® + 14541x^//® + 9149y® + 7639x//®)/(9594 + 7377y 
+ 15644y^ + 6261//® + 1988/ + 14942y® + 12768y® + //'^). 



(8) 
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A general form of defining equations of C'a.s.r curve is Equation (2). In this 
case, we have 

0 = 11654A + 6133A^ + 10293A® + 3017y + 463Xy + Y^ + 7669Z + 15127XZ 
0 = 15687A + 8029A^ + 10416A® + 9882^"^ + 14252y + 6982Xy + 9150A^y 
+ 4600Z + 6150AZ + YZ 

0 = 1362X + 11237A^ + 3867A® + 95X"‘ + 8346y + 9761Xy + 10084AV 

+ 5949X®y + 1677Z + 7169AZ + 831A^Z + Z'^. 

By the result of section 3.1, we guess that the above curve has a 

Jacobian of the order 

/i = 4394252339947x427379515481622744216694600721926448140291414819361 

over the finite field FpS for p = 16529. In fact, it is verified that h times a random 
rational point of the curve over FpS is equal to the unit element of the Jacobian, 
using the addition algorithm in [2]. 

Similarly, secure examples in genus 2 and 3 with Cat equations were found 
for 21 different levels N < 109. 
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Abstract. A general type of ray class fields of global function fields is 
investigated. The computation of their genera is reduced to the deter- 
mination of the degrees of these extensions, which turns out to be the 
main difficulty. While in two special situations explicit formulas for the 
degrees are known, the general problem is solved algorithmically. The 
systematic application of the methods described yields several new ex- 
amples of algebraic curves over F 2 , F 3 , F 4 , F 5 and F 7 with comparatively 
many rational points. 



1 Introduction 

The maximum uumber of Fq-ratioual poiuts ou a (smooth, projective, absolutely 
irreducible algebraic) curve Al|Fq of geuus g(X) = g defiued over the fiuite field 
Fq is usually deuoted by Nq{g). lu the early eighties, Serre [20,21,22] has writteu 
dowu formulas for Ng{l) aud Nq{2). 

Siuce the precise value of Nq{g) is quite difficult to determiue iu geueral, the 
work of mauy mathematiciaus has iustead led to large tables, such as [6] , giviug 
au iuterval for this quautity. The lower bouuds are usually realized by Abeliau 
coveriugs of small geuus curves, which are either giveu by explicit equatious 
(Hauseu aud Stichteuoth [7], [8], [24], vau der Geer aud vau der Vlugt [3], [4], 
[5], Niederreiter aud Xiug [12], [13], [15], Shabat [23], aud others) or obtaiued 
by class field theory or au equivaleut coustructiou (School [19], Tauter [11], 
Niederreiter aud Xiug [25], [13], [14], [16]). (Please uote that these refereuces are 
far from beiug complete.) The preseut paper, which adds to the secoud category, 
summarizes the author’s thesis [1], where all results stated here are proved iu 
detail. 

Siuce we employ ray class field exteusious, to the curve XjFq we associate the 
global fuuctiou field K = Fg(A). Its geuus gx equals g{X), aud coveriugs of X 
correspoud to field exteusious of K, the degree of the coveriug beiug the degree 
of the exteusiou. By coustructiou, F^ is the full coustaut field, i.e. is algebraically 
closed iu K, aud we express this iustauce by writiug K\¥q. 

A place of K, by which we meau the maximal ideal p iu some discrete valu- 
atiou riug of K, with (residue field) degree d = degp, correspouds to (a Galois 
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conjugacy class of) d points in X(Fgd), and each point on X having F^d as its 
minimal field of definition over F^ lies in such a conjugacy class. In particular 
K\¥q has Nk = |X(Fg)| rational places, i.e. places of degree 1. Throughout 
this paper, K\¥g is a global function field, and all algebraic extensions of K are 
assumed to lie in some fixed algebraic closure K of K 

2 Ray Class Fields 

We fix a non-empty set S of places of K, and denote the greatest common divisor 
of the degrees of its elements by d := gcdjdegp | p G S'}. Let m be an S-cycle, 
i.e. an effective divisor of K with support away from S. We consider the S-ray 
class field mod m, denoted K^, which is defined as the largest Abelian extension 
LlA" of conductor at most tn such that every place of S splits completely in L. 
These extensions occur e.g. in Ferret [17]. 

In the special case of m = o (the zero element in the divisor group). Kg is 
also known as the S-Hilbert class field (cf. Rosen [18]). We recall that the Galois 
group G{Kg\K) is isomorphic to the (ideal) class group Ci(Os) of the Dedekind 
ring Os consisting of all functions with poles only in S. 

Since S is non-empty, by class field theory is a finite (algebraic) extension 
of K. In fact, using Cebotarev’s Density Theorem, any finite Abelian L\K is seen 
to be equal to some K^. Here for m we can take the conductor of L\K, and S 
can always be chosen finite. Furthermore, the ray class fields satisfy the following 
properties. 

Proposition 1. Let S, d and m he as above, T another non-empty set of places 
of K and n a T-cycle. 

(a) The full constant field of Kft has decree d over F„, thus AT!?|F„d. 

(b) If S T and m < n, then C Ktf . 

(c) Lfg n where the minimum is taken coefficient wise. 

In terms of the ray class fields of K, we can write down the genus for any 
Abelian extension of K. 

Theorem 1. Let S, d, m = ^piripp be as above, and L an intermediate field 
of K'g\K. Then the genera gK and gr of K and L satisfy 

mp 

d. {g, - 1) = [L : K]{gK - 1+ n : K] degp . 

p n—1 

This formula can be proved either by applying Mobius inversion to the Conductor 
Discriminant Product Formula, as done by Cohen et al. [2] in the number field 
case, or by using Hilbert’s Different Formula, the Hasse-Arf Theorem and the 
connection between upper ramification groups and higher unit groups known 
from local class field theory (see [1]). 

We observe that computing the genus of ray class field extensions amounts to 
determining their degrees. This is easily done if S consists of just one place, but 
becomes much more intricate if we require more places to split. In the following 
section we indicate an algorithmic solution of this problem. 
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3 Computation of Degrees 



For simplicity we shall restrict to the case of ramification at only one place p 
of K\¥q (outside S), i.e. to tn = mp with m G Nq. Recall that, by definition, 
the residue field Fp of p satisfies [Fp : F^] = degp. We determine the degrees 
: iC] in three steps. 

First of all, from what has been said about the Hilbert class field, [Kg : K] 
equals the S-class number hs ■= \C£{Os)\- Its computation is connected with 
the problem of finding generators for the group Og of S-units, which in turn are 
needed for the other two steps. 

Indeed, by class field theory, the Galois group G{Kg\Kg) is isomorphic to 
the cokernel of the canonical group homomorphism Og F*. Since F* C Og, 

it follows that [Kg : Kg] divides ^ ^ . 

Similarly, the cokernel of Og n (1 + p) ^ (1 + p)/(l + p™) is isomorphic to 
G{K™^[Kg). According to the following theorem, its order can be determined 
for all m G N simultaneously. Let p be the characteristic of K, and define 

\a[p := min{p' [a<p^,l € No} 

for any real a > 0. 

Theorem 2. There are s := [S'! — 1 positive integers n\, . . . ,Us depending only 
on S and p such that 



[K^^ : K^g] = g(™-i)degp / 



m~ 

Hi 



for all m G N. 

The proof given in [1, p. 43] provides an algorithm for the computation of the 
numbers rii, . . . , n^. Since the order of these numbers is irrelevant, the behavior 
of the degrees can be summarized in the polynomial 

S 

i=l 



which is uniquely determined by S and p. Unfortunately we have no explicit 
formula for Ss,p except in some particular cases, which are treated in the next 
section. 



4 Rational Function Field 

Here we want to draw the attention to two special situations where the polyno- 
mial Ss,p can be given explicitly. 

Theorem 3. Let K he the rational function field over the prime field Fp, S a 
non-empty set of rational places of K and p a rational place of K not occurring 
in S; thus 0 < s := jS”] — 1 < p. Then 
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In particular 



K, and for m > s + 2 it follows that A'™’’ has exactly 



=1 + [K^^ :K]{s + l) 

rational places. The genus can be calculated by means of Theorem 1. As an 
example we have carried out these computations for p G {5,7} and different 
values of s and m. The results are displayed in the tables 1 and 2. 



Table 1. Ray class fields over F5 
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Table 2. Ray class fields over F7 
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Now let q = with e G N be an arbitrary power of the characteristic p 
again. Take Q := (1, . . . , g — 1} C Z as a set of representatives for the cyclic 
group Z /((7 — 1)Z ~ F*. Via this latter isomorphism, the group G := G(Fg|Fp) 
acts on Q. Clearly, two elements n,n' G Q lie in the same G-orbit Gn = Gn' iff 
n' = p^n mod q — 1 for some I G Nq. For n G N we define 






|Gn| if n G Q and n = minGn, 
0 otherwise. 



The following has been proved by Tauter [11]. 

Theorem 4. Let K he the rational function field over F^, p a rational place 
of K, and S the set consisting of the other q rational places. Then 63 , p = 

Let us set r := ^ or ytpq according to whether g is a square or not. By in- 
vestigating the numbers e„, we see that = K. Furthermore we obtain 
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two fields, namely and of degree r and rq over iC if q is a 

square, and p—1 fields with 1 < z < p of degree g* over K in case q is 

non-square. Lauter [10] has pointed out that the corresponding curves generalize 
certain families of Deligne-Lusztig curves. Here we want to write down defining 
equations for them, which might have been found by J. P. Pedersen before but 
without publishing. 

Proposition 2. Let K = Fq(x) with x an indeterminate over such that p is 
the pole of X, and let S (as in Theorem 4) consist of the remaining q rational 
places of K. 

(a) Assume that r := ^ G N and let y,z G K satisfy y^ + y = and 

z'^ — z = x’^^{x'^ — x). Then = K{y) and = K{y,z). 

(b) For r := ^Jpq G N let z/i, . . .,z/p-i G K satisfy y( — yi = x"l'^{x^ — s). 

Then = K{yi,. ..,yi) for z G {1, . . .,p- 1}. 

5 Tables 



Now we use ray class field extensions of small genus ground fields i^jF^ to pro- 
duce curves of higher genus with many rational points over F2, F3 and F4. Like 
in the tables [6] by van der Geer and van der Vlugt, we restrict ourselves to 
genus 5 < 50 and give a range (or the precise value) for Nq{g). The upper 
bound is taken from [6]. The lower bound is attained by a field L\¥g of genus 
9l = 9 with precisely that many rational places, and is set in boldface if our 
example actually improves the lower bound known before. The field L satisfies 
iCg™ C L C iC™’’ (thus has conductor mp) with m G N, p a place of K of 
degree d := degp, and S a non-empty set of rational places of K not containing 
P- 

The degrees : K] are computed as indicated in Sect. 3. In order to 

search through a large variety of possibilities for p and S, the algorithms have 
been implemented in KASH/KANT [9]. Then, by Theorem 1, the genus of L is 

, m— 1 

g^ = l + [L:K]{gK-l+^)--J2 ^ K] . 

n— 0 

Since the inertia degree of p in L is hs/hsu{f>}i L has 



Nl>[L-. K] |5| + 




if hs = hsu{v} and d = 1 
otherwise 



rational places. Here equality holds iff S already contains all rational places of K 
that split completely in L, which in our examples is always the case. Complete 
information on the precise construction of each field L occurring in the tables 3-5 
is given in [1]. 
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Table 3. Function fields over F2 with many rational places 



9 


N2{g) 


[L-.K] 


|S| 


m 


d 


9K 


6 


10 


10 


1 


2 


1 


1 


7 


10 


2 


5 


2 


6 


1 


8 


11 


2 


5 


10 


1 


2 


9 


12 


4 


3 


4 


2 


0 


10 


13 


4 


3 


4 


1 


2 


12 


14-15 


7 


2 


1 


6 


0 


14 


15-16 


15 


1 


1 


4 


0 


15 


17 


8 


2 


7 


1 


0 


16 


17-18 


2 


8 


14 


1 


5 


17 


17-18 


16 


1 


5 


1 


0 


19 


20 


4 


5 


2 


6 


1 


22 


21-22 


4 


5 


12 


1 


2 



9 


N2{g) 


[L-.K] 


|S| 


m 


d 


9K 


27 


24-25 


12 


2 


3 


2 


1 


28 


25-26 


8 


3 


7 


1 


2 


29 


25-27 


4 


6 


14 


1 


3 


30 


25-27 


4 


6 


12 


1 


4 


35 


29-31 


4 


7 


16 


1 


4 


37 


29-32 


4 


7 


14 


1 


5 


39 


33 


16 


2 


8 


1 


0 


41 


33-35 


8 


4 


6 


1 


4 


42 


33-35 


8 


4 


8 


1 


3 


44 


33-37 


8 


4 


11 


1 


2 


49 


36-40 


12 


3 


1 


6 


3 


50 


40 


8 


5 


2 


7 


1 



Table 4. Function fields over F3 with many rational places 



9 


N3(9) 


[L-.K] 


|S| 


m 


d 


9K 


5 


12-13 


3 


4 


2 


2 


1 


7 


16 


8 


2 


1 


4 


0 


9 


19 


3 


6 


5 


1 


2 


10 


19-21 


9 


2 


5 


1 


0 


14 


24-26 


3 


8 


5 


2 


2 


15 


28 


9 


3 


6 


1 


0 


16 


27-29 


9 


3 


3 


2 


0 


17 


24-30 


6 


4 


5 


1 


2 


19 


28-32 


3 


9 


12 


1 


3 


22 


30-36 


3 


10 


3 


5 


3 


24 


31-38 


3 


10 


14 


1 


4 


30 


37-46 


9 


4 


8 


1 


1 



9 


N3(9) 


[L-.K] 


|S| 


m 


d 


9K 


33 


46-49 


9 


5 


4 


1 


3 


34 


45-50 


9 


5 


3 


3 


1 


36 


46-52 


9 


5 


9 


1 


1 


37 


48-54 


24 


2 


1 


2 


2 


39 


48-56 


24 


2 


2 


2 


1 


43 


55-60 


9 


6 


11 


1 


1 


45 


54-62 


18 


3 


3 


2 


1 


46 


55-63 


27 


2 


6 


1 


0 


47 


54-65 


18 


3 


6 


1 


1 


48 


55-66 


9 


6 


11 


1 


2 


49 


63-67 


9 


7 


3 


4 


1 


50 


56-68 


28 


2 


1 


2 


2 
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Table 5. Function fields over F4 with many rational places 



g 


Na{9) 


[L-.K] 


|S| 


m 


d 


9K 


4 


15 


2 


7 


6 


1 


1 


5 


17-18 


4 


4 


6 


1 


0 


6 


20 


4 


5 


2 


3 


0 


8 


21-24 


2 


10 


6 


1 


3 


9 


26 


8 


3 


3 


1 


1 


10 


27-28 


12 


2 


2 


1 


1 


11 


26-30 


2 


13 


4 


3 


3 


12 


29-31 


4 


7 


8 


1 


1 


13 


33 


8 


4 


6 


1 


0 


14 


32-35 


8 


4 


2 


3 


0 


19 


37-43 


4 


9 


10 


1 


2 


20 


40-45 


8 


5 


3 


3 


0 


21 


41-47 


4 


10 


8 


1 


3 


22 


41-48 


4 


10 


10 


1 


3 


23 


45-50 


4 


11 


10 


1 


3 



9 


Niig) 


[L-.K] 


|S| 


m 


d 


9K 


24 


49-52 


4 


12 


10 


1 


3 


25 


51-53 


12 


4 


3 


1 


2 


27 


49-56 


16 


3 


6 


1 


0 


28 


53-58 


4 


13 


14 


1 


3 


31 


60-63 


15 


4 


1 


3 


2 


32 


57-65 


8 


7 


10 


1 


1 


33 


65-66 


16 


4 


7 


1 


0 


34 


57-68 


8 


7 


8 


1 


2 


36 


64-71 


8 


8 


3 


3 


2 


41 


65-78 


20 


3 


3 


1 


2 


43 


72-81 


24 


3 


2 


3 


0 


45 


80-84 


16 


5 


4 


2 


0 


47 


73-87 


8 


9 


12 


1 


2 


48 


80-89 


16 


5 


3 


3 


0 


49 


81-90 


8 


10 


10 


1 


3 
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Abstract. In this paper we introduce several new heuristics as to speed 
up known lattice basis reduction methods and improve the quality of 
the computed reduced lattice basis in practice. We analyze substantial 
experimental data and to our knowledge, we are the first to present a 
general heuristic for determining which variant of the reduction algo- 
rithm, for varied parameter choices, yields the most efficient reduction 
strategy for reducing a particular problem instance. 



1 Introduction 

A lattice is a discrete additive subgroup of M” generated by a lattice basis. Lat- 
tice basis reduction is the computation of lattice bases consisting of basis vectors 
which are as small in length as possible. The underlying theory has a long history 
starting with the reduction of quadratic forms and recently obtained general in- 
terest with the introduction of the LLL lattice basis reduction algorithm [13]. 
It spurred extensive research thus leading to the discovery of important connec- 
tions of lattice theory with other fields in mathematics and computer science. 
In particular, the progress in lattice theory has revolutionized combinatorial 
optimization [9] and cryptography (e.g., [1,2,6,8,11]). Nevertheless, despite the 
manifold results in theory and the availability of implementations of lattice ba- 
sis reduction algorithms in various computer algebra systems (e.g., LiDIA [14], 
Magma [15] or NTL [18]) there is still very little known about the practical 
performance and strength of lattice basis reduction algorithms. Thus, they are 
often underestimated as recent results show (e.g., on breaking cryptosystems 
using lattice basis reduction methods [16,17]). With this paper we close this gap 
by providing the results of analyzing extensive test data thus supplying useful 
facts on the practical performance of widely used lattice basis reduction algo- 
rithms. Moreover, we introduce newly-developed heuristics designed to improve 

* The research was done while the author was a member of the Graduiertenkolleg 
Informatik at the Universitat des Saarlandes, Saarbriicken (Germany), a fellowship 
program of the DFG (Deutsche Forschungsgemeinschaft). 
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known lattice basis reduction methods in practice and provide detailed test data 
on these new methods which are only implemented in LiDIA so far. We restrict 
the discussion in this paper to the LLL algorithm and its variants as the most 
well-known and most widely used lattice basis reduction methods in practice. 

The outline of the paper is as follows: In Section 2 we give a brief introduction 
to lattice theory by covering the basic terminology and stating basic auxiliary 
results in particular of the LLL algorithm, which is the first known polynomial 
time lattice basis reduction algorithm guaranteed to compute lattice bases con- 
sisting of relatively short basis vectors. We then focus on the Schnorr-Euchner 
algorithm [22] which provided the first essential improvement for making the 
LLL-reduction algorithm efficiently applicable in practice. In Sections 3 and 4 
we introduce (newly-developed) variants of the Schnorr-Euchner reduction algo- 
rithm (e.g., based on modular techniques) designed to achieve better run times 
and reduction results in practice than the classical Schnorr-Euchner algorithm, 
especially for large lattice bases or bases with large entries. The development of 
these new heuristics is motivated by the fact that because of the heuristics for 
preventing and correcting floating point errors, both the run time and the stabil- 
ity of the classical Schnorr-Euchner algorithm strongly depend on the precision 
of the approximations used. Hence due to stability reasons, for large lattice bases 
or bases with large entries, a high precision for the approximations has to be 
used in the classical algorithm thus causing a major loss in efficiency. Section 5 
is devoted to the description of suitable test series for testing the different re- 
duction algorithms (i.e., the classical Schnorr-Euchner algorithm as well as the 
variants presented in Sections 3 and 4) and the analysis of the corresponding 
substantial experimental data. Among other things, the analysis shows clearly 
the efficiency of the newly-developed algorithms. Furthermore, based on the data 
and the analysis, we present a general heuristic for determining which variant of 
the reduction algorithm, for varied parameter choices, yields the most efficient 
reduction strategy (with respect to run time or quality of the basis) for reducing 
a particular problem instance. 



2 Background on Lattice Basis Reduction 



In this section we present the basic definitions and results which will be used in 
the sequel. For more details and proofs we refer to [4,9,19,24]. In the following, 
let n,k G N with k < n. By j|6jj we denote the Euclidean length of the column 
vector b and for z € M, [zj stands for the closest integer to z. An integral 



lattice L C 



Z” is defined as L = 



{ ULl Xiki 



Xi G Z,i = 1, 






where 



are linearly independent vectors. We call B = (6^, . . . , &j.) G 
Z”^^ a basis of the lattice L = L{B) with dimension k. Obviously, a lattice has 
various bases whereas the dimension is uniquely determined. A basis of a lattice is 
unique up to unimodular transformations such as exchanging two basis vectors, 
multiplying a basis vector by —1 or adding an integral multiple of one basis 
vector to another one, for example. The determinant det(L) := j det(H^i?)j 2 
of the lattice L C Z” with basis H G Z”^^ is independent of the choice of the 
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basis. The Hadamard inequality det(L) < Y[i=i gives an upper bound for 
the size of the determinant of a lattice. Equality holds iff B is an orthogonal 
basis. Furthermore, the defect of B is defined as dft(i?) = Oti ll^ill- 

general, dft(i?) > 1 and dft(i?) = 1 iff i? is an orthogonal basis. Lattice basis 
reduction is a technique for reducing (possibly minimizing) the defect of a lattice 
i.e., it is a technique to construct one of the many bases of a lattice such that 
the basis vectors are as small as possible (by means of the Euclidean length) and 
are as orthogonal as possible to each other. The most well-known lattice basis 
reduction method is the so-called LLL-reduction [13]: 

Definition 1. For a lattice L C Z” with basis B = (5^, . . .,&j,) G Z”^^, corre- 
sponding Gram- Schmidt orthogonalization B* = (6 *, . . ., &^) G and Gram- 

Schmidt coefficients with 1 < j < z < fc, the basis B is called LLL-reduced if 
the following conditions are satisfied: 

iMijI < ^ /orl<j<z<fc (1) 

\\b* + iZi,i-i^r_if > forl<i<k. (2) 

The first property (1) is the criterion for size-reduction. The constant factor | 
in (2) is the so-called reduction parameter and may be replaced with any fixed 
real number y with i < y < 1. 

Theorem 1. Let L be a lattice in Z” and B = {b^, ... G be an LLL- 

reduced basis of L. Then, the following estimates hold: 

II61II •...•115,11 <2^('=-D/Met(L) (3) 

Ii5i|p < 2^“^||z;|p for all v£ L,v^ 0 (4) 

For any lattice LG IF with basis B = (5j^, . . ., 5,) G Z”^^, the LLL-reduction of 
the basis B can be computed in polynomial time. More precisely, the number of 
arithmetic operations needed by the LLL algorithm is O(fc^zzlogC'), and the in- 
tegers on which these operations are performed each have binary size 0{klogC) 
where C G M, C > 2 with ||5j|p < C for 1 < z < fc. Thus, from a theoretical 
point of view, the algorithm performs very well since it yields a reasonably good 
reduction result within polynomial time. In practice however, the classical algo- 
rithm [13] suffers from the slowness of the subroutines for the exact long integer 
arithmetic which has to be used in order to guarantee that no errors occur in 
the basis (thus changing the lattice) . Speeding up the algorithm by simply doing 
the operations in floating point arithmetic results in an unstable algorithm due 
to occurring floating point errors and error propagation. In [22], Schnorr and 
Euchner have rewritten the original LLL algorithm in such a way that an ap- 
proximation of the integer lattice with a faster floating point arithmetic is only 
used for the computation of the Gram-Schmidt coefficients {I < j < i < k) 
while all the other operations are done on the integer lattice using an exact 
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integer arithmetic. Moreover, Schnorr and Euchner have introduced heuristics 
for avoiding and correcting floating point errors, thus inventing a practical float- 
ing point variant of the original algorithm with good stability, which allows a 
tremendous speed-up in the computation of an LLL-reduced lattice basis. Nev- 
ertheless, for large lattice bases or bases with large entries, the algorithm still 
lacks of efficiency due to the fact that high precision approximations have to be 
used for stability reasons. 

Therefore, before focusing on the presentation and discussion of practical 
results achieved by using the Schnorr-Euchner reduction algorithm in different 
settings (Section 5), thus comparing theory and practical performance of this 
lattice basis reduction algorithm, we will in the following (Sections 3 and 4) 
first present (new) heuristics designed to further speed up the reduction (such 
that even larger lattice bases with bigger entries can be reduced in a reasonable 
amount of time) and improve the quality of the reduction results (i.e., computing 
reduced lattice bases consisting of shorter lattice vectors in comparison with the 
reduction results obtained by the classical Schnorr-Euchner algorithm). Due to 
the page limit we can only sketch the basic ideas and refer to [24] for a detailed 
description. 



3 Heuristics to Achieve an Additional Speed-Up 

In this section we will introduce heuristics designed to allow a speed-up of the 
computation in comparison to the classical Schnorr-Euchner algorithm [22] . The 
first heuristic in this setting, the so-called late size-reduction heuristic, is mo- 
tivated by the following observation: While at stage I {2 < I < k) of the LLL- 
reduction process the Gram-Schmidt coefficients {I < m < I — 2) have to 
be size-reduced only if the basis vectors 6; and bi_i are not swapped [13], in the 
classical Schnorr-Euchner algorithm always all the Gram-Schmidt coefficients 
k-i,m (1 < w < ^ — 1) are size-reduced. Therefore, a heuristic to speed up the 
LLL-reduction process in practice can be stated as follows: 

Heuristic 1 (Late Size- Reduction). Before checking the LLL condition {2) 
at stage I of the reduction process [22], size-reduce only the Gram-Schmidt co- 
efficient lapi-i- Perform the size-reduction of the other coeff dents yLpm with 
I < m < I — 2 only if neither the stage index has to he decreased {due to accu- 
mulated floating point errors) nor the basis vectors bi and bi_i have to swapped. 

While the late size-reduction heuristic centers on the time when the size-reduc- 
tions are performed, another new heuristic, called modified size-reduction heuris- 
tic, focuses on the way the size-reductions are done in order to speed up the 
reduction of a lattice basis: 

Heuristic 2 (Modified Size- Reduction). At the beginning of each size-re- 
duction step {see ]22J) check whether |["/iijJ| {f f j < i f k) is larger than 
a certain hound. If so, perform a correction step and simplify the actual size- 
reduction by approximating \ [/iijJ | with 2* where t = ]"log(| ["/iijJ |)J, thus replac- 
ing the original size-reduction with a fast shift operation. 
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Whereas the heuristics introduced so far simplify or eliminate unnecessary oper- 
ations, a different approach to reduce the overall run time can be taken by doing 
the computations on shorter operands thus allowing approximations with a lower 
precision than in the classical Schnorr-Euchner algorithm. The first heuristic in 
this category employs an iterative technique similar to the ones used in [12] and 
[21] for speeding up Euclid’s algorithm and the reduction of quadratic forms, 
respectively: 

Heuristic 3 (Iterative Heuristic). For reducing a given lattice basis, first 
work only with the leading digits of each entry of the basis vectors. Then, apply 
the performed reduction steps also to the original lattice basis and compute the 
final LLL-reduced lattice basis. 



The following theorem is a generalization of the idea presented in [20,23] to 
arbitrarily chosen lattice bases. It shows in detail how the described heuristic is 
formalized and how it can be iterated: 

Theorem 2. Let B = ( 6 ij) G ^ basis of lattice L and u,v G N be 

such that Llog 2 ( 6 ij)J + 1 < u • v {1 < i < n, 1 < j < k). Moreover, for 



l<i<n, l<j<fc and 1 < t < 



^et b['] = 



_Ki 

2”(“-0 



and bW = (bf]). 



let dW = be defined by + d['] and let E = 2Mk. With 

= Ik, = B<-^\ = LLL{C^*'>) = C^^^Tt, 

as well as E + then = LLL{B) = BT^^f {Tt is the 

transformation matrix computed in the course of the reduction process [13,22].) 



Thus, for reducing the basis B, the iterative reduction algorithm applies the 
classical Schnorr-Euchner algorithm u times to the generating systems . . ., 
In each iteration step, v additional digits of the original input data are 
included in the computation. Hence, the result of the last iteration step yields 
the LLL-reduction of the lattice basis B. 

The last heuristic presented in this section is based on modular computa- 
tions. From other areas in algorithmic number theory we know that the applica- 
tion of modular techniques has been instrumental in bringing about much more 
efficient solutions to well-known problems such as the computation of the deter- 
minant [4] or the Hermite normal form [7] of integer matrices. This is due to the 
fact that with modular techniques most of the computations can be performed 
on operands which are much smaller than the ones occurring in the conventional 
algorithms. The following considerations show how modular techniques can also 
be applied to the problem of computing an LLL-reduced lattice basis, thus al- 
lowing an improvement of the run time of the classical reduction algorithm: 



Theorem 3 ([7]). Let L C IF be an n-dimensional lattice and A = z ■ det(L) 
with z G I be a multiple of the lattice determinant. Then, Ae^ G L for 1 < i < n. 

Lemma 1. Let L C Z” 6e an n-dimensional lattice with basis B = {bi, . . . , b„) G 
jnxn A = z ■ det(L) where z G'l. Then, L{b ^, . . . , &„) = L{b^ mod A, . . ., 
mod A, Ae_i, . . ., 2\e„). 
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This leads to the following heuristic, which can be implemented in various 
ways [24]: 

Heuristic 4. At first reduce the basis of the n- dimensional lattice L C Z" mod- 
ulo A, a multiple of the determinant, thus obtaining a system of generating 
vectors {bi mod A,...,b^ mod A). Apply to that system of generating vectors 
a modular variant of the Schnorr-Euchner algorithm where additional modular 
operations are performed during the size-reduction process. Compute the LLL- 
reduced basis of the lattice L{bi, . . by applying the Schnorr-Euchner algo- 
rithm to the system of generating vectors consisting of the resulting vectors of 
the reduction with the modular Schnorr-Euchner algorithm as well as the vectors 
Ae^ , e^ . 

4 Heuristics to Achieve Better Reduction Results 

After introducing heuristics for speeding up the computation of a reduced lattice 
basis, we will now present two heuristics where the improvement of the quality 
of the reduction result is the main objective in their development, possibly even 
at the expense of the run time. The first heuristic, the so-called deep insertion 
heuristic, is due to Schnorr and Euchner [22] : 

Heuristic 5 (Deep Insertion). Eor checking the LLL-reduction condition {2) 
at stage I take into consideration not only the values and ||^|| {as it 

was done in the LLL algorithm and the classical Schnorr-Euchner algorithm) 
but also the earlier ||6*|| ’s with 1 < j < ^ — 2 by extending the exchange of bi 
and bi_i to a deep insertion step, i.e., inserting at the best possible position i 
within the index interval [1 , . . . , I — 1 ] . 

Applying this heuristic, short orthogonal vectors are found further left in the 
orthogonalization thus yielding shorter basis vectors in the reduced basis than 
in the case of using the classical Schnorr-Euchner algorithm. It has to be noted 
that only a certain amount of deep insertion steps can be performed in order to 
guarantee polynomial run time of the reduction process (for details see [22]). 

A new heuristic to achieve a better reduction result is motivated by the 
following example: 

/lO -9 18 \ 

Example 1. Let B = \ 0 10 45 I be a basis of a lattice L Q lA. The basis 

yiO 11 -12 j 

B is already LLL-reduced with reduction parameter y G (|, 1)> ll^ilP = 200, 

1 1 62 IP = 302 and ||^||^ = 2493. However, performing an additional size-reduction 
step even though [^3,2! = 0.5 (i.e., size-reduction does not change [^^3,21) results 
in a shorter basis vector 63 = (27, 35, —23)^ with ||63||^ = 2483 and thus in a 
better LLL-reduced lattice basis. 

Heuristic 6 (Special Case for /i). If \nij\ = 0.5 with 1 < j < i < k perform 
a size-reduction step iff\\b^ — sign{pLij)b^\\ < ||5j||. 
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5 Tests and General Heuristic 



Based on comprehensive tests, we will now analyze the practical performance of 
the classical Schnorr-Euchner algorithm as well as the algorithms implementing 
the (new) heuristics thus providing essential new insight into the practical per- 
formance of lattice basis reduction algorithm aside from the known theoretical 
results. Moreover, we present a general heuristics for the use of these lattice basis 
reduction algorithms. 

For the tests we have used three generic test classes, namely knapsack lattices, 
unimodular lattices, and random lattices. Knapsack lattices arise in the context 
of solving knapsack problems [6] and are of the form L{B) C where 



/ 2 0 0 

I 0 2 0 



0 1 X 

0 1 



B — (&i , . . . , 6„+i ) 



0 ••• 0 2 01 

0 0 ••• 0 2 1 

ailE a 2 W ■ ■ ■ a„-iW a„W SW 

0 0 ••• 0 0-1 

\ W W ■■■ W w ^wj 



(5) 



with positive integer weights at {1 < i < n), a, sum S' G N and W > ^/n. This 
test class has been chosen due to the fact that lattice bases in various contexts 
(e.g., finding a small root of univariate modular equations [5], gcd computations 
[10], factoring, Diophantine equations [13]) have the same structure as the bases 
of knapsack lattices. The unimodular lattices are generated by a unimodular 
basis B, i.e., B G with | det(i?)| = 1, the random (n x n)-lattices L C Z” 

are generated by a randomly chosen basis B G Z^”^”). In the sequel, for sim- 
plicity we shall use the notion “knapsack lattice bases” etc. instead of “bases 
of knapsack lattices” etc. even though we are aware of the fact that it is not 
absolutely correct in a mathematical sense. 

All tests have been done using the implementations of lattice basis reduction 
methods available in the computer algebra LiDIA [3,14]. While there are various 
implementations of the classical Schnorr-Euchner algorithm (e.g., in computer 
algebra systems such as LiDIA, Magma [15] and NTL [18]), so far implementa- 
tions of the heuristics presented in Sections 3 and 4 are only available in LiDIA. 

The tests have been performed on Sparc 4’s with 110 MHz and 32 MB main 
memory. For a detailed description of the test instances and the choice of the 
test parameters we refer to [24]. 



5.1 Tests of the Different Variants 

5.1.1 Performance and Quality. At first we focus on the general perfor- 
mance of the classical Schnorr-Euchner algorithm, the variant of doing deep 
insertions, the algorithm of applying late size-reduction, the algorithm using the 
modified size-reduction and the variant considering the special case = 0.5. 
The computations were done with reduction parameter y = 0.99 using doubles 
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for the approximations [22] . The tests have been performed for different choices 
of n (e.g., n = 10,..., 150) and various sizes of the entries (e.g., bit length 
6= 20,..., 300). 

The test results [24] show that for a fixed dimension n the run time of the 
algorithms increases as the density of the knapsack decreases. For a fixed density 
the run time for reducing knapsack lattice bases increases as the dimension 
increases. These characteristics are due to the fact that the run time of the 
algorithms depends both on the dimension of the lattice to be reduced and the 
size of the input data such that it increases as the dimension or the size of the 
entries of the lattice basis vectors grow. Furthermore, the reduction of knapsack 
lattice bases always results in a vast decrease of the average length of the basis 
vectors and the defect. 

For randomly chosen lattice bases the reduction time is relatively small (in 
comparison with the other test instances) and depends mainly on the dimension 
of the lattice. This is due to the fact that any randomly chosen lattice basis 
is already significantly reduced, thus the reduction will result only in a small 
decrease of the average length of the basis vectors. This can be explained by the 
observation that due to the Gaussian heuristic, the expected length of a smallest 
vector in a random lattice of dimension n with determinant A lies between 
and In the case of reducing unimodular lattice bases, the 

reduction time mainly depends on the dimension of the lattice. 

Checking whether an additional size-reduction step should be performed if 
l/iiji = 0.5 causes an increase of the run time but in general does not yield 
a shorter average length or a smaller defect. However, in the case of reducing 
knapsack lattice bases, the increase is negligible. In general, for random lattice 
bases the special case \ = 0.5 does not occur. This is due to the fact that 
only few reduction steps are performed anyway (in comparison with the other 
test instances) and therefore it is very unlikely that = 0.5 occurs. 

Except for random lattice bases the deep insertion mechanism causes a major 
increase in the run time but also results in a great improvement of the average 
length of the vectors of the reduced bases as well as a better defect for knap- 
sack lattice bases. In Figure I, the influence of the deep insertion mechanism is 
illustrated on example of the first five basis vectors of reduced knapsack lattice 
bases with n = 150. In the case of random lattice bases, only few deep insertion 
steps are performed since these lattice bases are already quite well reduced from 
the beginning. Thus, there is hardly any decrease of the defect or the average 
length in comparison with the results of the classical algorithm. Logically, for 
knapsack and unimodular lattice bases, the amount of deep insertions increases 
with the dimension of the lattice. This is due to the also increasing amount of 
reduction steps which simply implies a higher probability that deep insertions 
can be performed. 

As for the late size-reduction variant it turns out that this algorithm performs 
well for small-dimensional test lattices (n < 30) but lacks stability otherwise, 
even though provisions for correcting floating point errors and preventing error 
propagation were already taken. For stability reasons it seems to be crucial 
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that at stage I of the reduction process all Gram-Schmidt coefficients with 
1 < j < ^ — 1 are size-reduced before a possible step back might occur (due to a 
large size-reduction coefficient), thus allowing a faster decrease in the size of the 
intermediate results as it is the case in the late size-reduction algorithm. 

Applying the modified size-reduction algorithm for reducing the lattice bases 
causes no stability problems. But even though the size-reduction process was 
simplified, i.e., in the case of a large reduction coefficient, the original size- 
reductions were replaced with simple shift operations, it turned out that in most 
cases the heuristic does not improve the reduction time. This is due to the fact 
that in many cases, the shift operations are not accurate enough in a sense that 
after size-reducing the Gram-Schmidt coefficient is still far off from 

being less or equal to 0.5, thus causing additional operations. 

In summary, one may say that for reducing unimodular lattice bases neither 
the application of the deep insertion mechanism nor the checks on the special 
case =0.5 are useful since already the classical Schnorr-Euchner algorithm 
yields a minimal basis for unimodular lattices and none of the mechanisms yields 
an advantage with respect to the run time. For knapsack lattice bases the addi- 
tional checks in the case of = 0.5 make no big difference in the run time but 
might yield shorter basis vectors in some cases, thus supporting the application 
of this heuristic in practice. Using the deep insertion mechanism for reducing 
knapsack lattice bases has the disadvantage of increasing the run time but also 
the advantage of a decrease of the defect and average length of the basis vectors. 

5.1.2 Limits. In the following, we will concentrate on tests of the limits of 
the classical Schnorr-Euchner algorithm and how improvements can be achieved 
by using the newly-developed modular and the iterative variant of the classical 
reduction algorithm. In this context, limits are either meant to be the bounds 
at which the other variants begin to out-perform the classical algorithm or the 
bounds from which on xdoubles (floating point arithmetic with twice the preci- 
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sion of doubles) or even bigf loats (multi-precision floating point arithmetic, 
see also [14]) have to be used for doing the approximations in the classical 
Schnorr-Euchner algorithm in order to guarantee that the algorithm will ter- 
minate and yield an LLL-reduced lattice basis while in the case of the modular 
and iterative variants doubles are still sufficient for achieving the same result. 
It is important to know these limits since both the size of the input data as 
well as the approximations used affect the run time of the reduction algorithm 
fundamentally. 

In the first test scenario, we have applied the classical Schnorr-Euchner al- 
gorithm doing the approximations with doubles, xdoubles and bigf loats and 
the iterative algorithm with v = 0.256, v = 0.336, v = 0.56 and v = 0.756 
(doing the approximations with doubles) to knapsack lattice bases with n = 
100, 110, . . . , 200 and bit lengths 6 = n, 2n,4n. The reduction parameter was 
chosen as y = 0.99. 

In the second set-up, we were focusing on reducing lattice bases B G Z”^” 
with large entries where the corresponding lattice L{B) has a small determinant 
det(T) = A, i.e., n = 10, . . . , 100, 6 = 200, 400 and A e [1, 2^^]. The tests were 
performed using the classical Schnorr-Euchner algorithm, the modular variant 
Modular _1 and Modular _2 = Schnorr-Euchner (6 mod Z\, . . . , 6„ mod A, Ain) 
(doing the approximations by means of doubles and using reduction parameter 
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Fig. 2. Different Variants: Knapsack Lattice Bases (6 = 4n) 
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The tests on knapsack lattice bases show (see Appendix and [24]) that us- 
ing the Schnorr-Euchner reduction algorithm and doing the approximations by 
means of doubles works well for lattice bases with b = n and 6 = 2n. In the 
case of 6 = 4n and starting at n = 130, the approximations using doubles are 
no longer exact enough, thus resulting in a non-reduced basis. At the same time. 





New Results on Lattice Basis Reduction in Practice 



145 



using xdoubles is not sufficient either. This is due to the fact that xdoubles 
have twice the precision of doubles but no larger exponent [14]. Consequently, 
not only the precision but also the size of the exponent of the approximation 
is crucial for the stability of the algorithm. Using bigfloats with four times 
the precision of doubles and an enlarged exponent increases the reduction time 
considerably but yields a correctly reduced lattice basis. A major improvement 
can be achieved by using the iterative algorithms (see Figure 2). 

The results of the iterative variant, choosing v = 0.256, v = 0.336 and 
V = 0.56 {v is the amount of additional digits of the original input data which are 
included in each new iteration step) show that doing the approximations with 
doubles is sufficient for all test classes. However, since several lattice basis re- 
ductions have to be done in the course of the iterative algorithm, the iterative al- 
gorithm does not out-perform the classical Schnorr-Euchner algorithm until the 
Schnorr-Euchner algorithm requires xdoubles (bigfloats) for the approxima- 
tions while the iterative variant still works with doubles (xdoubles). These be- 
havioral characteristics of the iterative variant and the classical Schnorr-Euchner 
algorithm (in combination with doubles, xdoubles and bigfloats for the ap- 
proximations) are illustrated in Figure 2 for knapsack lattice bases with 6 = An. 

In Figure 2 it can be seen that as long as the approximations are done with 
doubles the run time of the iterative implementation is about twice that of the 
Schnorr-Euchner algorithm. Furthermore, the data for the iterative lattice basis 




Fig. 3. Different Variants: Lattices with Small Determinant 



reduction algorithms show that the reduction time decreases as v increases since 
a large v requires fewer iterations. At the same time, the computations have to 
be performed on larger operands, thus possibly causing the same problems as 
in the case of using the classical Schnorr-Euchner algorithm. For example, for 
V = 0.75, n > 170 and 6 = 4n it is no longer sufficient to do the approximations 
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by means of doubles. On the other hand, for large lattice bases with huge 
entries, the run time decreases as v decreases. In this case, the advantage that 
the computations can be done on small operands predominates the disadvantage 
that many iterations have to be performed. Hence, these observations show that 
the size of v and thus the amount of iterations has to be chosen skillfully in order 
to obtain the best possible results. 

The modular variants out-perform the classical Schnorr-Euchner algorithm 
for increasing b. This is due to the fact that using the modular variants, the 
operands on which the computations are performed are much smaller in size than 
in the case of applying the classical Schnorr-Euchner algorithm. For increasing b 
this advantage compensates the disadvantage of the additional computations in 
the case of the modular variants, necessary in order to guarantee that the correct 
LLL-reduced basis is computed in any case. The performance of the algorithms 
is impressively demonstrated in Figure 3 for lattices with a small determinant 
and b = 400. Figure 3 also shows that the differences in the run times of the 
modular variants are small. Consequently, after the initial modular reduction of 
the lattice only few additional modular reductions have to be performed in the 
course of the size-reductions in the algorithm Modular_l. 

To sum up, one may say that for large dimensions or large entries of the lattice 
bases, the newly-proposed modular and the iterative variants out-perform the 
classical Schnorr-Euchner algorithm. 

5.2 Additional Tests 

In addition to the tests described so far, experiments with varied reduction 
parameters respectively series of reduction parameters and varied scalar products 
have been performed. Moreover, the use of performing the reductions based on 
the Gram matrix instead of the original basis of the lattice was tested [24] . 

In summary, it may be said that for achieving the best reduction results 
in the case of knapsack lattice bases, it is recommended to apply a sequence 
of reductions with varied reduction parameters. For reducing unimodular and 
random lattice bases in general it is sufficient to use the reduction parameter 
y = 0.75. By using a weighted scalar product to reduce a given lattice basis 
it is possible to decrease the row sum norm of a particular row of that lattice 
basis significantly. Moreover, the Gram heuristic is only advisable for reducing 
unimodular lattice bases. 

5.3 General Heuristic 

Based on the comprehensive analysis of the test results we can now deduce 
heuristics for the general use of the different variants of the Schnorr-Euchner 
algorithm in order to achieve the best possible reduction results or to minimize 
the necessary reduction time: 

In the case of reducing knapsack lattice bases, it has to be noted that in 
general these two goals cannot be achieved at the same time. Generally, the 
following heuristic is suggested: 
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Heuristic 7. In order to minimize the run time for reducing small knapsack 
lattice bases use the classical Schnorr-Euchner algorithm in combination with a 
sequence of reduction parameters doing the approximations with doubles. For 
large lattice bases or lattice bases where the size of the entries is large {more 
than 4-00 bits) use the iterative algorithm also in combination with a sequence of 
reduction parameters. In order to maximize the quality of the reduction of bases 
corresponding to knapsack lattices apply the deep insertion heuristic doing the 
approximations with doubles. For large lattice bases with huge entries {more 
than 400 bits) use the iterative algorithm in combination with the deep insertion 
mechanism. 

Since the tests have shown that randomly chosen lattice bases are already sig- 
nificantly reduced, it is suggested to LLL-reduce them in the following way: 

Heuristic 8. For reducing randomly chosen lattice bases use classical Schnorr- 
Euchner algorithm with y = 0.75 doing the approximations by means 0 / doubles. 
If the determinant is known for large {n x n) -lattice bases, or {n x n) -lattice 
bases with large entries, apply the modular reduction variant. Otherwise, use the 
classical Schnorr-Euchner algorithm and adjust the approximations if necessary. 

In the case of unimodular lattice bases, the following heuristic should be applied: 

Heuristic 9. For reducing unimodular lattice bases compute the corresponding 
Gram matrix and apply the Schnorr-Euchner reduction algorithm with reduction 
parameter y = 0.75 and doubles for the approximations. For large lattice bases 
with huge entries adjust the approximations if necessary. 

For a given lattice basis which does not necessarily belong to any of the classes 
discussed so far we suggest the following method for reducing the given basis: 

Heuristic 10. If the given lattice basis is sparse, apply the proposed heuristics 
for knapsack lattice bases. In the case of a dense lattice basis, proceed as in the 
case of randomly chosen lattices. 

If the run time is not of importance, the quality of the reduction of a lattice basis 
can be even further improved by repeatedly sorting and mixing up the reduced 
basis (by performing weight-reductions, permuting the basis randomly or using 
so-called Hadamard transformation matrices) and reducing the basis again [24] . 

6 Conclusions 

In this paper, we have presented various new heuristics and analyzed a com- 
prehensive series of tests on the practical performance of the different variants 
of the Schnorr-Euchner algorithm. Based on these results, we have introduced 
heuristics for the general application of these lattice basis reduction algorithms 
designed to minimize the reduction time or achieve a best possible reduction 
result in practice. For any given reduction algorithm (e.g., Korkine-Zolotarev- 
reduction [19]), we believe that in order to draw similar conclusions about the 
best reduction strategy, experimentation and analysis similar to that presented 
herein must be performed. 
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Appendix: Tests 

In the following, we provide some of the test data [24] for the tests described and 
analyzed in Section 5. The general notation used is as follows: 

determinant determinant of the lattice 

defect defect of the lattice basis 

av. length average length of the base vectors 

factor factor = {density)~^ 

time in (m)s time in (milli)seconds needed for reducing the lattice 

reduction steps number of size-reductions performed 
correction steps number of approximations performed 

swaps number of exchanges 

step backs number of occurring decreases of the stage index 

exact SP number of exactly computed scalar products 

LLL original Schnorr-Euchner algorithm - doing the approximations 

with doubles (y — 0.99 unless otherwise stated) 

LLL_2 original Schnorr-Euchner algorithm - doing the approximations 

with xdoubles {y — 0.99) 

LLL_4 original Schnorr-Euchner algorithm - doing the approximations 

with bigf loats having four times the precision of doubles as 
well as an enlarged exponent {y — 0.99) 

LLL_Iterative_0.50 iterative variation of the Schnorr-Euchner algorithm with 

V = 0.506 and doing the approximations with doubles (y = 0.99) 
Modular_l modular variation of the Schnorr-Euchner algorithm doing the 

approximations with doubles (y — 0.99) 

Modular_2 = Schnorr-Euchner_Generate(6j mod A, . . . ,b^ mod A, Ain) 

with y — 0.99, A — det(L{B)) and doing the approximations 
with doubles 

Knapsack knapsack lattices 

Random_Det lattices generated using a modification of the LiDIA function 
randomize_with_det 

Note that the figures provided are rounded values corresponding to the original results. 
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Abstract. The baby-step giant-step algorithm, due to Shanks, may be 
used to solve the discrete logarithm problem in arbitrary groups. The 
paper explores a generalisation of this algorithm, where extra baby steps 
may be computed after carrying out giant steps (thus increasing the 
giant step size). The paper considers the problem of deciding how many, 
and when, extra baby steps should be computed so that the expected 
cost of the generalised algorithm is minimised. When the logarithms are 
uniformly distributed over an interval of length n, the expected cost of 
the generalised algorithm is 6% lower than that of Shanks (achieved at 
the expense of a slightly larger worst case cost) . In some situations where 
logarithms are far from uniformly distributed, any baby-step giant-step 
algorithm that computes all its baby steps before taking a giant step 
must have infinite expected cost, but the generalised algorithm has finite 
expected cost. The results are heuristic, but are supported by evidence 
from simulations. 



1 Introduction 

The classic baby-step giant-step algorithm due to Shanks (see, for example, Co- 
hen [1]) makes use of a time-memory trade-off to search an interval of length 
n for a discrete logarithm using only 0{^/n) operations. In the standard appli- 
cation, we assume that the distribution of answers (logarithms) is uniform over 
the interval. 

This paper considers the following situation. Suppose a baby-step giant-step 
algorithm is to be used to find a discrete logarithm, and that the logarithm is 
taken from a known distribution that is not necessarily uniform. (This distribu- 
tion could be rigorously derived, but it could also be found by experimentation 
or arise from heuristic results.) How can we minimise the expected number of 
operations of the algorithm? For which distributions is it a good idea to compute 
more baby steps after some giant steps have been carried out (so increasing the 
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giant step size)? Our motivation is from recent papers of Stein and Teske [5,6], 
that give an algorithm to find the divisor class number of a hyperelliptic func- 
tion field. Their approach is essentially a baby-step giant-step algorithm that 
searches an interval for a discrete logarithm that is approximately normally dis- 
tributed. We will comment briefly on this situation in the penultimate section 
of the paper. However, in general this paper emphasises the study of baby-step 
giant-step algorithms rather than specific applications. 

Our goal is to design a baby-step giant-step algorithm that will return an 
integer picked from a given distribution using the smallest expected number 
of operations. We will consider a one sided distribution, where small integer 
values are more likely to occur than larger values. This can be easily adapted to 
two sided distributions such as those considered by Stein and Teske — see the 
penultimate section of the paper. 

The theoretical and experimental data both suggest that the best way of 
computing baby steps depends on the hazard function of the distribution (see 
the next section for a definition of this concept). For non-increasing distributions 
on the non-negative integers, we conclude the following. If the distribution has 
expected value E and increasing hazard rate, then '/E baby steps should be 
computed, and then all the giant steps should be carried out. If, however, the 
distribution has decreasing hazard rate, extra baby steps should be computed 
after some giant steps have been taken — see the next section for details. 

The arguments in this paper are not rigorous. In particular, we are very 
cavalier with error terms. Instead, we support our arguments with experimental 
data. 

The remainder of the paper is organised as follows. Section 2 presents the 
generalisation of the baby-step giant-step algorithm we will consider. Arguments 
are given as to why this algorithm is optimal. Section 3 lists several distributions 
and calculates the theoretically optimal algorithm for each of them. Finally, 
Section 4 presents the simulation results that support the arguments in Section 2. 



2 A Baby-Step Giant-Step Algorithm and Its 
Optimisation 

This section analyses a baby-step giant-step algorithm that finds an integer s 
taken from a distribution on the non-negative integers. Rather than computing 
all the baby steps at the beginning, the algorithm computes extra baby steps 
(thus increasing the giant step size) after each unsuccessful giant step. The num- 
ber of baby steps that are computed is controlled by a function b. The question 
this paper addresses is: For a given distribution, what choice of b minimises the 
expected number of operations of the algorithm? 

Let 6 be a monotonic increasing positive-integer valued function on the non- 
negative integers. In the algorithm below, the variable x holds the smallest in- 
teger not checked at any stage; the variable y holds the number of baby steps 
that have been computed. 
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1. Set a; = 0 and y = 0. 

2. Compute b{x) — y extra baby steps, so that a total of b{x) baby steps have 
been computed. Set y = b{x). 

3. Perform a giant step (of size b{x)) to scan the interval [x,x+ b{x) — 1]. If s 
lies in this interval, output s and stop. 

4. Set a; = a; -I- b{x). Return to step 2. 

If it is known that s < n for some integer n, and if b is the constant function 
with value [-v/nli have the original algorithm of Shanks. If b is any function 
such that b{2{i -|- I) -I- — 1)) = i + 2 for all i G {0, 1,2,.. .}, we have the 

algorithm recently proposed by Terr [7]. 

In order to choose a sensible function b for a particular application, we re- 
strict the distributions we consider to those that are non-increasing. More pre- 
cisely (and to fix notation), let po^pi, . . . be non-negative real numbers such that 
Assume that pi > pi+i for all i. We consider the distribution asso- 
ciated with the probabilities pi. Let X he a, random variable taking value i with 
probability pi, and let E be the expected value of X. 

We now define a function b that comes close to minimising the expected 
number of operations required by the algorithm above. (We note that most of 
the information encoded in the function b is not used. For example, none of 
the values 6(1), 6(2), . . . , 6(6(0) — 1) affect the algorithm. We assume that these 
unused values of 6 are chosen so as to make 6 a ‘natural’ monotonically increasing 
function that interpolates the points that matter.) Define a function m, from the 
non-negative integers to the reals by 



= ( 1 ) 
V Pk 

If m is monotonic increasing, let 6 be a function such that 6(fc) is an integer as 
close as possible to m{k), subject to b{k) > 1. If m is monotonic decreasing, let 
6 be a constant function equal to an integer which is approximately \/E. If m is 
not monotonic, then a good choice for 6 consists of segments that approximate 
m (since 6 is monotonic increasing, m must be monotonic increasing on these 
segments), together with horizontal lines that join these segments. (In this last 
circumstance, there does not seem to be a simple rule to determine 6 in general. 
However, given a distribution, it is not difficult to search for an optimal value of 
6 from functions of this form.) 

This choice of the function 6 is partly justified by simulations; see the final 
section of the paper. The rest of this section attempts to give a conceptual 
justification for this choice of 6, by analysing a second algorithm. This second 
algorithm would not be used in practice. However, its expected cost is easier to 
analyse, and simulations show that the two algorithms have comparable costs. 
Before carrying out this analysis, we digress to discuss the relationship between 
the choice of 6 given above and a concept in statistics known as the hazard rate. 

If the distribution does not decay too rapidly, the ratio inside the square root 
in (1) is well approximated by (XXfc Pi) /Pk- This quantity is known as the Mills’ 
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ratio, and its reciprocal is known as the hazard rate (or failure rate) at k; see 
Johnson, Kotz and Kemp [2, p.lll]. If X is the random variable corresponding 
to the probabilities pi, the hazard rate at k measures the likelihood of X lying 
in a small interval beyond k, given that X > k. (It is used in contexts where 
the random variable models the likelihood of some machine component failing 
within a time interval.) We then have an intuitive explanation for the form of 
the function b above. For if the hazard ratio is high at a point x, and we know 
that s ^ [0, a; — 1], then it is likely that s will occur soon, and so few baby steps 
should be computed. If the hazard ratio drops as x increases, more baby steps 
should therefore be computed at each stage, and so b should be an increasing 
function. Distributions of this type are known as Decreasing Hazard Rate (DHR) 
distributions; they tend to be distributions with large tails. If the hazard rate 
increases (so the distribution is an Increasing Hazard Rate (IHR) distribution), 
the optimal number of baby steps at a point drops. But in this case, since baby 
step computations cannot be taken back once they are computed, the best we 
can do is to compute no extra baby steps, and so 6 is a constant function. This 
argument gives some reasons why we might expect b to depend on the hazard 
rate, but does not reveal the precise form of b. We now give an argument that 
indicates this more precise form. 

We briefly describe our second algorithm. Let ft- be a positive integer. Divide 
the non-negative integers into intervals /q, /i, . . . of length ft, where Ij = [jft, (j-l- 
l)ft— 1]. Deflne Pj = -Pj is the probability that s lies in Ij. To 

scan each interval Ij, the algorithm computes some baby steps, increasing the 
giant step size to ft(jft) steps. The algorithm then uses \h/b{jh)~\ giant steps to 
test whether s G Ij. 

We may regard this second algorithm as an approximation to the first, where 
the number of baby steps is only updated when ft integers have been scanned 
since the last update. For all the approximations we will make to be reasonable, ft 
should be of a moderate size, slightly larger than the square root of the expected 
value of a random variable taking the value k with probability pk, say. 

The expected number of operations needed to And s using this second algo- 
rithm is approximately 



OO 






[b{jh) + 


■ ft ■ 


_ 1 _ 


■ ft ■ 


J- ... J- 


ft 




ft(0) 




ft(ft) 




^((j - m 


) 



This is a lower bound for the expected number of operations. An upper bound 
is the same expression with a term \h/b{jh)~\ added to the sum. A more precise 
estimate would replace this term with {E{X\X G Ij) — jh)/b{jh), where the 
random variable X is such that Pr(A = i) = pi. 

Rearranging this sum, and ignoring the rounding errors caused by requiring 
the number of giant steps on each interval to be integers, we And that the total 
expected cost of the algorithm is 



OO / 

^ Pkb{kh) + 

k=0 V 



hET=k+iPA 

b{kh) j ■ 



(2) 
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If we wish to minimise the expected cost of the algorithm, we need to choose the 
function b so as to minimise the expression (2). Define m{kh) to be that value 
of h{kh) that minimises the fc-th term of (2). So m is a function from the set of 
non-negative multiples of h to the non-negative integers, and 

OO \ 

E ’ (3) 

=k+l ) 

if this expression is an integer, or is one of the two integers closest to this value 
otherwise. (Note that if Pfc = 0 then this value is not well defined; but in this 
case the value of h{kh) does not affect the running time of the algorithm. So 
when Pk = 0, we set m{kh) = m{{k — l)h) without loss of generality.) 

If the function m defined by (3) is monotonic increasing, setting b{kh) = 
m{kh) for all k minimises the expected cost of the algorithm. But for many dis- 
tributions, m is not monotonic increasing and so we cannot use m{kh) to define 
b{kh) at all values of kh. However, if 6 is a function that minimises the cost of the 
algorithm it is not difficult to see that either b{kh) = m{kh), or b{kh) > m{kh) 
and b{kh) = b{{k—l)h), or b{kh) < m{kh) and b{kh) = b{{k+l)h). Thus a min- 
imal function b is made up of horizontal line segments, together with segments 
of the function m defined by (3). In particular, when m is monotonic decreas- 
ing (which is the case for a great many sensible probability distributions), the 
minimising function 5 is a constant function. Interpreted in algorithmic terms, 
this means that all the baby steps should be computed at the beginning, before 
any giant steps are carried out — this is the approach of the original algorithm 
of Shanks. When b is constant, the value of b giving minimal expected cost is 
about y^E{X), where X is the random variable such that Pr(AT = i) = pi for all 
i. To see this, when b{k) = b for some integer b we may write the cost function 
(2) as 

f:(pkb+i{p,+p,+,+---))=b+f:fp,. 

k=0 ^ ' k=0 

This last sum is approximately E{X)/b provided that h is small enough, since 
the event that s lies in Ik occurs with probability Pk and this event occurring 
means that s is approximately hk. Since the minimum value of b+E(X) /b occurs 
when b = y^E{X), choosing b to be this value should minimise the expected cost. 




3 Example Distributions 

This section considers various distributions, and determines a suitable strategy 
for computing baby steps in each case. 



3.1 The Uniform Distribution 

Suppose s is uniformly distributed throughout the interval [0, n— 1], so 

p. = / n if * < ^ > 

10 otherwise . 
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This is the situation most often considered. In this case, m{k) is approximately 
^/n — k when k < n, and m{k) = 0 when k > n. Since m is monotonic decreasing, 
the optimal choice for 6 is a constant function approximately equal to a/ n/2. So 
the original approach of Shanks, computing all the baby steps first, is the best 
possible here. (However, Shanks’ version of computing approximately baby 
steps yields a worse average-case running time but a better worst-case running 
time.) 



3.2 The One Sided Normal Distribution 



Suppose that s follows a discrete approximation to one side of a normal distri- 
bution of mean 0 and standard deviation a. So 



Pi ^ 






The normal distribution is an IHR distribution (see, for example, Kececioglu [3, 
p.362], and so the optimal value for b should be a constant function with value 




3.3 The Discrete Pareto Distribution 

Let Pi = c/{i + l)*^, where d> 1 and where c is chosen so that Y^^^Pi = 1, i.e. 
c = 1/C((i), where C( ' ) denotes the Riemann zeta function. (This distribution is 
also referred to as the Zipf distribution, or the Riemann zeta distribution.) 

Then the function m{k) defined by (1) is approximately \Jkjid— 1). Since 
this function is monotonic increasing, we set b{k) = maxjl, [m(fc)]} in this case. 
For this choice of b{k), our algorithm becomes similar to an algorithm recently 
proposed by Terr [7]. Note that when d < 2, an algorithm that computes all its 
baby steps at the beginning has an infinite expected cost, while for the algorithm 
with b{k) = m{k) has infinite expected cost only when 1 < d < 3/2. 



3.4 The Weibull Distribution 



The discrete Weibull distribution is approximately given by 





where the positive real numbers (3 and 77 are respectively the shape parameter 
and the scale parameter. We only consider the case when 0 < /3 < 1 — the case 
when /3 = 1 is the geometric distribution (see the next subsection) and when 
(3 > 1, the distribution is not decreasing (the pt rapidly climb to a maximum 
value, before falling gently towards zero) . 

The hazard function of the distribution is approximately \{k) = f3/rj-(k/Tf)^~^ , 
which is strictly decreasing for 0 < /3 < 1. The expected value E{X) is approx- 
imately rj r{l/P + 1), where F denotes the gamma function. Now, m(fc) is ap- 
proximately l/i/A(fc) so we set b{k) = maxjl, m{k)} for fc > 1, and 5(0) = 5(1). 
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3.5 The Geometric Distribution 

Let Pi = qp'‘, where p + q = 1. The function m defined by (1) is approximately 
\fpjq, a constant function. Note that E{X) = p/q in this case, and so the value 
for b obtained by using the function m is equal to the value of b that is obtained 
by minimising the expected cost over all constant functions. 



3.6 The Split Uniform Distribution 

Suppose that the interval [0, n — 1] is divided into two parts [0, ^ — 1] and [£,n—l] 
in each of which s is uniformly distributed, i.e. 

{ PA ifi < i , 

Pb ii £<i <n , 

0 otherwise , 

where pA, Pb and I are such that pA > Pb and ipA + (n — £)pb = 1, i-e., 
Pb = (1 — £pA)l{n — tj. Then m{k) = ^JljpA ~ (^ + 1) if fc < ^ and m(fc) = 
y/n — (fc -b 1) if < fc < n, so that m is neither decreasing nor increasing. Since 
m is decreasing on the intervals [0,^— 1] and n — 1], the function b should 
be constant on these two intervals. Thus b is determined by 6(0) and b{£). So a 
good choice of 6 is computed by minimising the expected cost over all choices of 
6(0) and 6(f); in other words, by minimising 

Hence, the best values for 6(0) and 6(f) should be approximately 

6(0) = + {n- £)pb^ and 6(f) = £) , 



which yields an increasing function if pA > 2 In. If pA < 2/n, the cost function 
is minimised by a constant function, with value yj E{X), where 



f2 

E{X) = -PA + 




3.7 Two Sided Distributions 

The distributions we have considered have all been non-increasing, starting at 0. 
We may easily modify our treatment to deal with two sided distributions such as 
discrete approximations to the normal distribution. In this case, our algorithm 
should start at the peak value of the distribution, making giant steps in both 
directions away from the mean. In the case of a symmetrical distribution about 
0, the cost function may be approximated by 

26 

Pkb{kh) H 

k=0 




E OO p 

j=k+l 

b{kh) 



(4) 
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where here Pk is the probability that s S /fc U —Ik- This changed cost function 
leads to an altered function m{k), that differs by a factor of -\/2 from the original. 
The optimal value for a constant function b is equal to y^2E{\X\). 

To take a specific example, if s is approximately normally distributed with 
mean 0 and standard deviation a, we find that the optimal choice of 6 is a 

constant function with value . 



4 Experimental Results 

We presented two algorithms in Section 2. The first algorithm, the ‘practical’ 
algorithm, has a running time that is difficult to analyse precisely. The second 
algorithm, the ‘theoretical’ algorithm, was used to derive an optimal choice for 
the baby-step function b. It is important to verify experimentally that the run- 
ning times of the two algorithms are comparable when a theoretically optimal 
choice of b is used, and that the optimal value for b predicted by the theory ac- 
cords well with practice. This section contains the experimental results we have 
obtained. 

Subsection 4.1 gives the mean cost of the practical and theoretical algorithms, 
when s is taken from some of the distributions considered in Section 3. Their 
costs are found to be comparable. Baby step functions corresponding to the 
algorithms of Shanks and Terr are also included for the purpose of comparison. 

Subsection 4.2 attempts to find the best choice of the function b experimen- 
tally. The optimal choice for b via experiment is shown to agree well with the 
choice predicted by the theory. 

For all our experiments, we used the computer algebra system LiDIA [4] . 



4.1 Testing the Theoretically Optimal Baby-Step Functions 

This subsection compares the mean costs of the practical and theoretical algo- 
rithms on a range of distributions, where the baby step function b is taken to be 
the theoretically best choice. Costs of the algorithms of Shanks and of Terr are 
also included, by considering appropriate baby step functions b. 

To this end, we conducted the following experiment. We considered ap- 
proximately N values of s, taken from various distributions po,pi, . . . ,Pn-i on 
[0,n — 1]. In our experiment, we took n = 100000 and N = 2 ■ 10®. For each 
distribution, we carried out the following steps: 

1. Simulate the probability distribution on [0,n— !].• 

For 0 < z < n, let Si be the nearest integer to PiN. Let N = Si, and 

let Pi = Si/ N. Compute the expected value E{s) = ^Pi- 

2. Compute the theoretically optimal baby-step function: 

If the distribution is IHR, let 6opt(0) be the closest integer to \jE{s), and 
let bopt{k) = bopt{k — 1) for 1 < fc < n. 

If the distribution is DHR, use (I) with Pi,pk replaced by Pi,pk to find m(fc) 
(0 < fc < rz) and let bopt{k) be the integer closest to m{k) subject to the 
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conditions that bopt{k) > 1 and that 6opt is not decreasing. In the case of the 
split uniform distribution, choose the function b as given in Subsection 3.6. 
For the explicit baby-step functions see further below. 

3. Compute Shanks-type baby-step functions: 

Define bs,n{k) = 0 < k < n. This is the baby-step function which is 

canonically used in baby-step giant-step algorithms. This function does not 
depend on the probability distribution. 

If the distribution is not IHR, we also define the function 6s , e as a constant 
function with value yjE{s). This represents the best choice of baby-step 
function, if one is restricted to computing all the baby steps before a giant 
step is carried out; if the distribution is IHR, this function is identical with 

6opt- 

4. Compute Terr’s baby-step function: 

Let bT{k) = 2 for 0 < fc < 4 and, for fc > 4, let brik) = j -\- 2 where 
j = j{k) is the uniquely determined integer such that 2(j -|- 1) -I- ^j(j — 1) < 
k < 2(j -I- 2) -I- ij(j -I- l)).The function bT^k) essentially grows as v^. For 
example, we have the following values: 



k 


0 


4 


7 


11 


16 


22 




106 


121 




1036 


1082 




10012 


10154 




99682 


brik) 


2 


3 


4 


5 


6 


7 




15 


16 




46 


47 




142 


143 




447 



The practical algorithm with this choice of baby-step function is essentially 
identical to the algorithm of Terr. 

5. For each baby-step function above, determine the mean cost of the practical 
algorithm: 

For each z € [0, rz — 1] with sz yf 0 we count the numbers of baby steps and 
giant steps needed to scan the interval [0, z] using the practical algorithm, 
and multiply the respective results by s,. We add these results up for all z 
and divide the respective sums by N. This gives the average numbers of baby 
steps and giant steps. The mean cost is just the sum of these two numbers. 

6. For each baby- step function above, determine the mean cost of the theoretical 
algorithm: 

Let h be the integer nearest to \jE{s). Then we proceed analogously to the 
practical algorithm. Notice that, however, we do this only for Terr’s baby- 
step function and, if the distribution is not IHR, for the theoretically optimal 
function; for all Shanks-type functions, the theoretical and the practical al- 
gorithms are identical. 

The Explicit Probability Distributions and their Theoretically 
Optimal Baby-Step Functions. 

Recall that n = 100000 in all our experiments. 

The uniform distribution on [0, rz— 1] has expected value zz/2, so the theoretically 
optimal baby-step function is given by 6opt(fc) = 224 for all k. 

When considering the one-sided normal distribution, we work with a = 25000. 
On [0, zz — 1] we find that E{s) = 19941.4, so the theoretically optimal baby-step 
function is given by 6opt(fc) = 141 for all k. 
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Table 1. Testing the 6’s. n = 100000. 





1 b 


aver. ^ bs 


aver. # gs 


aver. # (bs+gs) 


Uniform distribution: E 


(s) = 49999.5 


Pr. = Th. 


optimal 


224.000 


224.712 


448.712 


Pr. = Th. 


Shanks, 317 


317.000 


159.225 


476.225 


Pract. 


Terr 


299.130 


299.138 


598.269 


Theor. {h = 224) 


Terr 


297.908 


396.789 


694.698 


One sided normal distribution: a = 


25000, E{s) 


= 19941.4 


Pr. = Th. 


optimal 


141.000 


142.920 


283.920 


Pr. = Th. 


Shanks, 317 


317.000 


64.395 


381.395 


Pract. 


Terr 


184.805 


184.812 


369.617 


Theor. {h = 141) 


Terr 


183.511 


244.451 


427.962 


Pareto distribution: d = 1.2, E(s) = 2453.9 


Pract. 


optimal 


27.363 


25.675 


53.038 


Theor. {h = 50) 


optimal 


25.441 


31.739 


57.180 


Pr. = Th. 


Shanks, 50 


50.000 


50.164 


100.164 


Pr. = Th. 


Shanks, 317 


317.000 


8.777 


325.777 


Pract. 


Terr 


29.016 


28.858 


57.875 


Theor. {h = 50) 


Terr 


27.371 


36.198 


63.569 


Weibull distribution: 0 = 0.5, rj = 


10000, E{s) = 


= 12882.4 


Pract. 


optimal 


107.772 


104.297 


212.069 


Theor. {h = 114) 


optimal 


105.769 


111.365 


217.135 


Pr. = Th. 


Shanks, 114 


114.000 


114.427 


228.427 


Pr. = Th. 


Shanks, 317 


317.000 


42.009 


359.009 


Pract. 


Terr 


123.536 


123.560 


247.096 


Theor. {h = 114) 


Terr 


121.468 


166.963 


288.432 


Geometric distribution: cr = 0.9999, E{s) — 9994.5 


Pr. = Th. 


optimal 


100.000 


101.435 


201.435 


Pr. = Th. 


Shanks, 317 


317.000 


32.999 


349.999 


Pract. 


Terr 


126.283 


126.295 


252.579 


Theor. {h = 100) 


Terr 


124.918 


166.920 


291.838 


Split uniform distribution: £ = 20000, pa = 1/25000; E(s) = 19999.5 


Pract. 


optimal 


137.952 


139.861 


277.813 


Theor. {h = 141) 


optimal 


137.598 


139.939 


277.537 


Pr. = Th. 


Shanks, 141 


141.000 


143.338 


284.338 


Pr. = Th. 


Shanks, 317 


317.000 


64.579 


381.579 


Pract. 


Terr 


175.519 


175.528 


351.047 


Theor. {h = 141) 


Terr 


174.181 


235.003 


409.184 



For the Pareto distribution, we choose d = 1.2, which yields E{s) = 2453.9. The 
theoretically optimal function bopt{k) looks as follows: 



k 


0 


2 


3 


5 


8 




107 


119 




1006 


1048 




10036 


10263 




19546 




39172 


Hk) 


3 


4 


5 


6 


7 




21 


22 




56 


57 




137 


138 




166 




184 



Here, 5opt is a step function, and we always tabulate the least value k for which 
^opt assumes the indicated value. In particular, for all 39172 < k < n we have 
boptik) = 184. 
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Table 2. Theoretical versus practical algorithm: varying h. 



II b 


aver. ^ bs 


aver. # gs 


aver. # (bs-|-gs) 


Uniform distribution: E(s) = 49999.5 


Pract. 


Terr 


299.130 


299.138 


598.269 


Theor. {h = 224) 


Terr 


297.908 


396.789 


694.698 


Theor. {h = 100) 


Terr 


297.812 


340.653 


638.465 


Theor. {h = 50) 


Terr 


297.836 


318.762 


616.598 


Theor. {h = 20) 


Terr 


297.924 


306.895 


604.819 


One sided normal distribution: a 


= 25000, E0 


i) = 19941.4 


Pract. 


Terr 


184.805 


184.812 


369.617 


Theor. {h = 141) 


Terr 


183.511 


244.451 


427.962 


Theor. {h = 100) 


Terr 


183.469 


225.983 


409.453 


Theor. {h = 50) 


Terr 


183.428 


204.192 


387.621 


Theor. {h = 20) 


Terr 


183.505 


192.415 


375.920 


Pareto distribution: d — 1.2, E{s) = 2453.9 


Pract. 


optimal 


27.363 


25.675 


53.038 


Theor. {h = 50) 


optimal 


25.441 


31.739 


57.180 


Theor. {h = 20) 


optimal 


26.396 


28.750 


55.146 


Pract. 


Terr 


29.016 


28.858 


57.875 


Theor. {h = 50) 


Terr 


27.371 


36.198 


63.569 


Theor. {h = 20) 


Terr 


27.955 


31.885 


59.840 


Weibull distribution: 0 = 0.5, r/ 


= 10000, E{s 


= 12882.4 


Pract. 


Terr 


123.536 


123.560 


247.096 


Theor. {h = 114) 


Terr 


121.468 


166.963 


288.432 


Theor. {h = 100) 


Terr 


121.597 


161.398 


282.995 


Theor. {h = 50) 


Terr 


122.011 


141.490 


263.501 


Theor. {h = 20) 


Terr 


122.221 


130.488 


252.710 


Geometric distribution: cr = 0.9999, E{s) 


= 9994.5 


Pract. 


Terr 


126.283 


126.295 


252.579 


Theor. {h = 100) 


Terr 


124.918 


166.920 


291.838 


Theor. {h = 50) 


Terr 


124.925 


145.333 


270.259 


Theor. {h = 20) 


Terr 


124.931 


133.676 


258.607 


Split uniform distribution: £ = 20000, pa = 1/25000; E(s) = 19999.5 


Pract. 


Terr 


175.519 


175.528 


351.047 


Theor. {h = 141) 


Terr 


174.181 


235.003 


409.184 


Theor. {h = 100) 


Terr 


174.224 


216.587 


390.812 


Theor. {h = 50) 


Terr 


174.127 


194.828 


368.955 


Theor. {h = 20) 


Terr 


174.189 


183.076 


357.265 



For the Weibull distribution, we choose (3 = 0.5 and rj = 10000. We find that 
E{s) = 12882.4, so the theoretically optimal function bopt{k) is a step function 
that takes the following values (tabulating as before): 



k 


0 


2 


3 


4 


5 


6 




104 


113 




1045 


1100 




10340 


10702 




35884 


b{k) 


14 


17 


19 


20 


21 


22 




45 


46 




79 


80 




135 


136 




166 



(In particular, bopt{k) = 166 when 35884 < k < n.) 
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Table 3. Testing the 6’s. n = 100000. With shortcut 





1 b 


aver. # bs 


aver. # gs 


aver. # (bs-|-gs) 


Uniform distribution: E 


(s) = 49999.5 


Pr. = Th. 


optimal 


223.750 


223.712 


447.462 


Pr. = Th. 


Shanks, 317 


316.499 


158.225 


474.725 


Pract. 


Terr 


299.130 


298.138 


597.269 


Theor. {h = 224) 


Terr 


297.908 


395.789 


693.698 


One sided normal distribution: a = 


25000, E{s) 


= 19941.4 


Pr. = Th. 


optimal 


140.678 


141.920 


282.599 


Pr. = Th. 


Shanks, 317 


315.386 


63.395 


378.782 


Pract. 


Terr 


184.805 


183.812 


368.617 


Theor. {h = 141) 


Terr 


183.511 


243.451 


426.962 


Pareto distribution: d = 1.2, E(s) = 2453.9 


Pract. 


optimal 


27.017 


25.881 


52.898 


Theor. {h = 50) 


optimal 


24.963 


30.739 


55.702 


Pr. = Th. 


Shanks, 50 


23.013 


49.164 


72.177 


Pr. = Th. 


Shanks, 317 


91.717 


7.777 


99.494 


Pract. 


Terr 


28.820 


27.858 


56.678 


Theor. {h = 50) 


Terr 


27.175 


35.198 


62.373 


Weibull distribution: 0 = 0.5, 77 = 


10000, E{s) = 


= 12882.4 


Pract. 


optimal 


107.756 


105.219 


212.975 


Theor. {h = 114) 


optimal 


105.544 


110.365 


215.909 


Pr. = Th. 


Shanks, 114 


106.764 


113.427 


220.191 


Pr. = Th. 


Shanks, 317 


282.521 


41.009 


323.53 


Pract. 


Terr 


123.536 


122.56 


246.096 


Theor. {h = 114) 


Terr 


121.468 


165.963 


287.432 


Geometric distribution: a — 0.9999, E(s) — 9994.5 


Pr. = Th. 


optimal 


99.502 


100.435 


199.937 


Pr. = Th. 


Shanks, 317 


312.029 


31.999 


344.028 


Pract. 


Terr 


126.283 


125.295 


251.579 


Theor. {h = 100) 


Terr 


124.918 


165.920 


290.838 


Split uniform distribution: £ = 20000, pa = 1/25000; E(s) = 19999.5 


Pract. 


optimal 


137.657 


138.861 


276.518 


Theor. {h = 141) 


optimal 


137.303 


138.939 


276.242 


Pr. = Th. 


Shanks, 141 


140.605 


142.338 


282.943 


Pr. = Th. 


Shanks, 317 


314.996 


63.579 


378.575 


Pract. 


Terr 


175.518 


174.528 


350.047 


Theor. {h = 141) 


Terr 


174.181 


234.003 


408.184 



For the geometric distribution, we work with p = 0.9999, and we find that 
E{s) = 9994.5. We calculate that the theoretically optimal baby-step function 
is defined by bopt{k) = 100 for all k. 

Finally, in the case of the split uniform distribution, we work with i = 20000, 
and PA = 1/25000. Then E{s) = 19999.5 and the optimal baby-step function is 
such that 6opt(fc) = 122 for 0 < fc < 20000 and bopt{k) = 200 for 20000 < k < n. 

Notice that in all distributions above, the parameters have been chosen so 
that maxjz ; Si ^ 0} = n — 1. 
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The results for this experiment are shown in Table 1. We see that in all cases, 
we get the best performance for both the practical and the theoretical algorithm 
if we use 6opt- In particular, the differences between the data for 5opt and for 
^S, 3 i 7 are quite impressive, especially in the case of the Pareto, the Weibull, 
the geometric and the one-sided normal distributions. While 6s , e still yields an 
acceptable or good performance, this is not the case for the Pareto distribution, 
where the use of a non-constant function 6 seems to be particularly practical. 

Although the theoretical and practical algorithms have comparable costs for 
a theoretically optimal choice of 6, we notice a discrepancy between the perfor- 
mances of other baby-step functions. This phenomenon is particularly striking 
in the case of Terr’s function, and is can be explained as follows: To search the 
interval [0, 6 — 1], we use giant steps of length 6t(0) = 2 in the theoretical algo- 
rithm. Thus, to find x in [0, 6 — 1], we use two baby steps and x/2 giant steps. 
To search the whole interval [0, 6—1], we use two baby steps and (6— l)/2 giant 
steps of length two. Only after that, the giant step size is increased to 6t(6), 
etcetc. This leads to a considerably increased number of giant steps, compared 
with the practical algorithm where the baby step size is adjusted after each giant 
step. The effect of this can be reduced if we decrease the size of 6, as the results 
in Table 2 show. 

Nevertheless, the theoretically optimal baby-step functions still yield the best 
performances, both for the theoretical and the practical algorithms. 



The Shortcut. If equality checks are cheap, what is the case in the majority of 
applications, the baby-step giant-step method usually is applied with a “short- 
cut”: Instead of computing all 6(0) baby steps first and then using the first giant 
step to check whether i e [0, 6(0) — 1], one checks for each newly computed baby 
step whether i has been found already. This means that one finds all i < 6(0) by 
performing just i+1 baby steps and equality checks, and no giant step. This tech- 
nique is particularly favourable for all Shanks-type functions. To check whether 
our theoretically optimal baby-step functions still give the best performances, we 
have implemented the shortcut in both the theoretical and practical algorithms. 
The results are shown in Table 3, where we see that indeed, the performance of 
6s, E and 6s , 317 improves, where the most dramatic improvement we observe in 
the case of the Pareto distribution. 



4.2 Experimental Finding of Best Baby-Step Functions 

In this section, we try to find best baby-step functions 6 experimentally. For 
this, we work with the same probability distributions as above, and we restrict 
ourselves to monotonically increasing step functions 6 that are constant on the 
five intervals [j • 20000, (j-k 1) • 20000 [, 0 < j < 4. We use the practical algorithm 
with the shortcut; by doing this, we risk some discrepancy with our theoretically 
optimal function, but this is the algorithm that would be used in practice, after 
all. 
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Table 4. Optimal step functions (5 steps). With shortcut. 



Distribution 


6(0) 


6(20000) 


6(40000) 


6(60000) 


6(80000) 


aver. #(bs+gs) 


Uniform 












447.179 














447.189 




225 


226 


226 


226 


226 


447.198 




231 


231 


231 


231 


231 


447.204 




224 


225 


225 


225 


225 


447.205 




224 


224 


224 


224 


224 


447.249 


One-s. normal 


142 


142 


142 


142 


142 


282.611 




141 


141 


141 


141 


141 


282.612 




142 


142 


142 


142 


143 


282.612 




141 


141 


141 


141 


142 


282.613 




142 


142 


142 


142 


144 


282.613 


Pareto ^ 


71 


167 


167 


167 


167 


52.898 

59.910 




71 


166 


166 


166 


166 


59.910 




71 


165 


165 


165 


165 


59.911 




71 


164 


164 


164 


164 


59.912 




72 


167 


167 


167 


167 


59.912 


Weibull ^ 


106 


155 


155 


155 


155 


212.975 

216.734 




107 


156 


156 


156 


156 


216.738 




107 


153 


153 


153 


153 


216.740 




106 


156 


156 


156 


156 


216.741 




107 


154 


155 


155 


155 


216.742 


Geometric ^ 




100 


100 


100 


100 


199.946 




100 


100 


100 


100 


102 


199.947 




100 


100 


100 


101 


101 


199.947 




100 


100 


100 


100 


103 


199.947 




100 


100 


100 


101 


102 


199.947 


Split uniform 


124 


194 


194 


194 


194 


276.265 




124 


195 


195 


195 


195 


276.265 




124 


196 


196 


196 


196 


276.265 




124 


197 


197 


197 


197 


276.266 




124 


198 


198 


198 


198 


276.267 




122 


200 


200 


200 


200 


276.422 



For each distribution, we proceed in several rounds, where with each round 
we converge on a good step function. In each round, we allow seven different 
values for the steps, which gives 462 distinct increasing step functions with five 
steps, and we keep track of those 20 combinations of values which yield the 20 
lowest average costs. (The costs are computed in the same way as in the previous 
section.) In the beginning the seven values are uniformly spread over the interval 
[0, max{-y/n, 2y/E{s)}], and based upon the best values of the previous round, 
we choose the seven values for the next round, so that eventually we end up with 
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what we hope are the best choice of values. A selection of our results - the best 
5 choices each together with the respective average costs - is shown in Table 
4. There, an arrow marks the theoretically optimal function; since the optimal 
functions for the Pareto and the Weibull distributions can not be written in that 
form, we simply copied the corresponding performance data for the optimal 
functions from Table 3. We see that in the case of the one-sided normal and 
the geometric distributions, the best step function and the theoretically optimal 
function (almost) coincide. In the case of the uniform and the split uniform 
distributions, these two function differ slightly. However, the difference in the 
average costs for the best step function and the theoretically optimal function 
is very small! Even in the case of the Pareto and the Weibull distributions, 
where the theoretically optimal function is much more complex, the best step 
functions yield average costs that are only slightly higher. Finally, it may be 
interesting to note that for the Pareto distribution from Section 4.1 we have 
6opt(0) = 3, 5opt(20000) = 166 and 6opt(40000) = 5opt(60000) = 5opt(80000) = 
184, while for the Weibull distribution we have 6opt(0) = 14, 5opt(20000) = 153 
and 6opt(40000) = 6opt(60000) = 6opt(80000) = 166. 



5 Conclusion 

We have examined, in theory and practice, the question of which strategy of 
computing the baby step set in the baby-step giant-step method yields the best 
average performance, given the probability distribution of the solution. Our re- 
sults can be used to considerably speed up the baby-step giant-step method, and 
to save storage space as well. 

In particular, our findings show that the original method of computing 
baby steps at the very beginning, and no other baby steps later (where n is 
the length of the interval in which the solution lies) is not optimal for any of 
the standard distributions, even the uniform distribution. Here, optimality is 
measured in terms of minimising the expected number of operations. Indeed, if 
the probability distribution of the solution has increasing hazard rate (which is 
the case for the uniform distribution, the one-sided normal distribution and the 
geometric distribution) and expected value E, the best average performance is 
obtained when about \/E baby steps are computed first, and then giant steps 
are computed until the solution is found. In particular, this means that for IHR 
distributions, the only information needed to execute the optimal strategy is the 
expected value. On the other hand, if the distribution has decreasing hazard 
rate (such as the Pareto distribution or the Weibull distribution), the optimal 
strategy consists in computing some baby steps, then some giant steps, then some 
more baby steps and so on, where the exact optimal number of steps depends 
on the whole sequence (pi) of probabilities. However, if this information is not 
available, applying the same strategy as for IHR distributions still gives much 
better results than the original version. Finally, it is worth noting that Terr’s 
algorithm is close to optimal for the Pareto distribution. 
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Abstract. In a paper of Kraus, it is proved that for 

p > 17 has only trivial primitive solutions, provided that p satisfies 
a relatively mild and easily tested condition. In this article we prove 
that the primitive solutions of with p — 4,5,7,11,13, 

correspond to rational points on hyperelliptic curves with Jacobians of 
relatively small rank. Consequently, Chabauty methods may be applied 
to try to find all rational points. We do this for p = 4, 5, thus proving that 
x^ + y^ = z'^ and a;® + y® = z^ have only trivial primitive solutions. In 
the process we meet a Jacobian of a curve that has more 6-torsion at any 
prime of good reduction than it has globally. Furthermore, some pointers 
are given to computational aids for applying Chabauty methods. 



1 Introduction 

In this paper we consider for given p = 2,3,... the Diophantine equation 

+ y^ = zP, x,y,z€ Z, gcd(x, y, z) = 1. (1) 

To emphasise that we look at solutions with gcd(a;, y, z) = 1, we refer to such 
solutions as primitive solutions. 

First, we review what is known. This equation is a special case of the gener- 
alised Fermat equation x'^ + y^ = z* . For fixed exponent triples r, s, t, a theorem 
of Darmon and Granville (see [8]) gives a classification of what the solution set 
looks like. If we apply their result to our equation, we get that for each p > 4, 
x^ + y^ = zP has only finitely many primitive solutions. If we assume the ABC- 
conjecture (see [23]), then we even get that for p big enough, + y^ = zP only 
has the trivial primitive solutions, i.e. with xyz = 0. This has led (together with 
the lack of counterexamples) to the very bold conjecture 

Conjecture 1 (Tijdeman, Zagier, Beal Prize Problem) Let x, y, z, r, s, 

t he positive integers with r, s,t > 2. If x'^ + y^ = z* then x, y, z have a factor in 
common. 

which even has a reward attached to its resolution (see [13]). 

For p = 2, the result by Darmon and Granville does not give information. In 
fact, there are infinitely many primitive solutions to -I- = 2 ^. A paper by 

Beukers (see [2]) guarantees that these solutions can be finitely parametrised. 
That means that one can give a finite number of polynomial solutions to x^+y^ = 

* funded by NWO grant “Groot project getaltheorie” . 
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such that each primitive solution can be obtained from one of the polynomial 
solutions by specialisation. In Lemma 1 we give these parametrisations. 

For p = 3, the Darmon and Granville result is also inconclusive. However, 
Euler and maybe even Fermat already proved that the equation 
has only trivial rational solutions. 

For p > 4, Darmon and Granville predict that there are only finitely many 
primitive solutions and it is generally believed that they are all trivial. Once we 
verify this for p = 4, it suffices to prove triviality only for prime p, since any 
composite p > 4 is either divisible by 4 or by a prime number> 3. This justifies 
that the exponent is tendentiously designated by p, hinting at primality. In this 
light, one may consider 4 as a composite number with primal tendencies. 

For 17 < p < 10000, Kraus (see [12]) has proved that there are no nontrivial 
primitive solutions, assuming the Taniyama-Weil conjecture that is now consid- 
ered proved by many. His proof resembles Frey’s and Ribet’s construction of a 
nonmodular elliptic curve assuming the existence of a nontrivial solution. For 
this, however, he needs the existence of a prime number with certain properties. 
While it seems plausible that such a prime exists for any p, Kraus failed at prov- 
ing so. Therefore, he checked the condition for all prime numbers in the given 
range individually. 

It is the purpose of this paper to sketch how the cases with p < 17 can be 
dealt with and carry out the procedure in a couple of cases. We will prove 

Theorem 1 Integer solutions to x^+y^ = z^ with xyz yf 0 have gcd(a;, y,z) > 1. 

Theorem 2 Integer solutions to x^+y^ = z^ with xyz yf 0 have gcd(a;, y,z) > 1. 

Alternatively, one may try to extend the method of Kraus to smaller p. 
In his paper, he assumes there is a nontrivial solution and constructs a non- 
GM elliptic curve from it. He then analyses the Galois-representation on the 
p torsion and concludes it is not modular. One step consists of showing that 
the representation should be irreducible. Since the curve constructed by Kraus 
has a rational 2-torsion point, the curve would correspond to a rational point 
on the modular curve Xo(2p) if the representation were reducible. Following 
[9], such curves are either singular or GM for prime p > 7. The curve Ao(lO), 
however, has genus 0 and has infinitely many rational points. It may be possible 
to prove irreducibility in this case using other arguments, however. Although the 
details are not (yet) available in the literature, it seems doable to extend Kraus’s 
methods to p = 5, 7, 11, 13 (see [11]). The number 4 still has enough composite 
features to completely break down Kraus’s method for p = 4, however. 

2 Parametrising Curves; Chabauty Methods 

If we use the word eurve, we mean a smooth projective geometrically irreducible 
variety of dimension 1. We will often work with singular, planar models of these 
curves. Some of these models will have singularities at their unique point at 
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infinity. If the smooth curve has only one point there, we will refer to that as oo 
and if it has two, we call them 00 “'" and 00 “ (arbitrarily but fixed). 

In this section, we will construct hyperelliptic curves over Q such that a 
primitive solution of corresponds to a rational point on one of the 

curves. Furthermore, we will give an estimate for the Mordell-Weil rank of the 
Jacobians of these curves and sketch how one may proceed in finding all rational 
points on these curves. The sketched method is not an algorithm, but in practice 
these methods often work. 

Suppose that we have x,y, z € Z with gcd(a;, y, z) = 1 and + y^ = z^. 
Note that gcd(a: + y, x'^ — xy + | 3 (an elementary resultant computation). 

Therefore, we have tG{— 1,0,1} and zi, Z 2 G Q such that 

x + y = 3^P, 
x"^ — xy + y^ = 3~*Z2, 

Z = ZiZ2- 

We solve for y in the first equation and substitute the value in the second equa- 
tion and divide by z\^ . This gives an equation that is of degree p in p- and 

quadratic in -%■. For each solution and given t, this gives us a rational point on 

•^1 

a fixed curve Cp^t in the following way. 




For odd p > 5, the curves Cp^t are of genus (p — l)/2 and also have a model 
of the form + Ap^f By Faltings’ theorem, we have that the number 

of rational points on a curve of genus > 2 is finite. Finding those points or, if 
you have them, proving that the list of points is complete, is a different matter. 
Faltings’ proof is not constructive, so it is of little help. 

There is an earlier, partial proof by Chabauty (see [6]) that uses a construc- 
tion that, if adapted in a proper way, might yield sharp bounds on the number 
of points. Suppose we have a (smooth) curve C of genus g over Q that has a 
known rational point Pq G C(Q). 

The Jacobian variety of C is an abelian variety over Q of dimension g. 
That means that it is a complete variety with a point O G JJ(Q) together with a 
morphism J Y. J ^ J over Q that defines a group operation on the points of J 
with O as a neutral element. The points of J coincide with the degree 0 divisor 
classes Pic°(C) and the fact that C(Q) yf 0 guarantees that f7(Q) — Pic°(C)(Q), 
i.e. the group of degree 0 divisors over Q modulo linear equivalence. 

Also, the map P 1 — > [P — Pq] gives an injective morphism C ^ J over Q. 
Therefore, we can consider C as a subvariety of J over Q. As such, we have 
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C(Q) C >7(Q). Chabauty’s method applies to any curve in an abelian variety, 
so for the sequel it is sufficient to consider an abelian variety of dimension g 
over Q with a curve C as a subvariety over Q. 

The set of rational points (Q) forms a finitely generated abelian group, the 
Mordell-Weil group. Therefore, there exist an r G Z>o and a finite group T such 
that f7(Q) ~ Z’’ 0 T. The number r is called the rank of J"(Q). 

Let p be a rational prime. Then f7(Qp) is a p-dimensional p-adic analytic 
abelian Lie-group. Such groups are locally isomorphic to (Zp)®. The topological 
closure of a finitely generated subgroup of rank r in such a variety will be of 
dimension < r. Therefore, if the Mordell-Weil rank r of J"(Q) is smaller than 
g, then the topological closure J (Q) of the Mordell-Weil group will be a proper 
analytic subvariety of J'(Qp). Since both C(Q) C J{Q) and C(Q) C C(Qp) C 
f^(Qp)> we see that 

C(Q) C C(Qp) n ■ 

The right hand side is the intersection of analytic subvarieties. If is indeed 
the Jacobian of C, then Chabauty has proved that C(Qp) does not have dimen- 
sion 1 intersections with proper analytic subvarieties of j7(Qp). Therefore, the 
intersection is of dimension 0 and, since j7(Qp) is compact, is finite. 

This proves that C(Q) is finite and that #C(Qp) n J'(Q) gives an upper 
bound for ^C{Q). In practise, it turns out that p can often be chosen such that 
the bound is sharp. Furthermore, if one can find Gi, . . . , Gm G i/(Q) such that 
(Gi, . . . , Gm) = then one can often approximate the points in C(Qp) n 

J7(Q) well enough to count them. 

Thus, to be able to apply this method to Cp^t{Q), we must estimate the 
Mordell-Weil ranks of the corresponding Jacobians. A theorem of Stoll (see [19] 
and [20]) helps. 

Let I be an odd prime, let A be a non-zero, 2Lth power free integer prime to I 
and let C be the smooth, complete curve with an affine model = X’’ + A. Let 
Q be a primitive fth root of unity and define qi{A) := {A^~^ — l)/l. We assume 
that —Aj^~^ is an odd, positive integer in which each prime factor occurs to 
an odd power. 

Define 

, / J \_j{l — 1)J — (— if A ■ qi{A) modi is a nonzero square in F/, 

^ ’ ( [|(l — 1)J otherwise. 

Theorem 3 (Stoll) LetC he the smooth, complete curve with model + 

A satisfying the conditions above. If the ideal class number o/Q(vG4, Q) is prime 
to I, then the rank of the Mordell- Weil group of the Jacobian of C is hounded 
above by di{A). 

It should be noted that Stoll’s original theorem gives a more precise statement 
for many more values of A, but this result suffices for us. The proof is based on a 
descent argument utilising the C/-action on the Jacobian, described by Schaefer 
(see [17] and [16]). 
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In order to apply the theorem to our situation, we have to check that the 
ideal class number of ^p) = Q(C 3 p) is not divisible by p. We do this using 

lower bounds on discriminants together with a little bit of class field theory. 

Let K he a number field. Let L\(iC) be the discriminant of the ring of integers 
of K. We define the root discriminant by rd(iC) := If p divides the 

ideal class number of K, then there is a degree p unramified relative extension 
L of K. Such an extension has [L : Q] = p\K : Q] and rd(L) = rd(iC). One can 
compute lower bounds on rd(iC), increasing in \K : Q] (see [15]). Thus, if rd(iC) 
is small enough, this puts a bound on the ideal class number of K. 

So, if the ideal class number of Q(C 3 p) is divisible by p. then there is a 
number field L with [L : Q] = 2p{p — 1) and rd(L) = rdQ(C 3 p). In [14] we find 
the following lower bounds on rd(L). 



p 


rd(Q(C3p)) 


[L-.Q] 


lower bound on rd(L) 


5 


5.79 


40 


12.96 


7 


8.77 


84 


15.87 


11 


14.99 


220 


18.59 


13 


18.18 


312 


> 19.23 


17 


24.66 


544 


< 23 (> 26.48 under GRH) 



So, obviously, for p = 5,7, 11, 13 such L cannot exist. Therefore, Theorem 3 
applies to Cp^t and we find 



P 


t 




genus(Cp,t 


) P 


t 




genus (Cp,t) 


5 - 




1 


2 


11 - 




3 


5 


5 


0 


1 


2 


11 


0 


3 


5 


5 


1 


0 


2 


11 


1 


3 


5 


7 - 


-1 


1 


3 


13 - 


-1 


3 


6 


7 


0 


1 


3 


13 


0 


2 


6 


7 


1 


1 


3 


13 


1 


2 


6 



So, we have proved that for p = 5,7, 11, 13, Chabauty methods may be applied 
to bound the number of primitive solutions to = z^. 

Amusingly, the root discriminant argument breaks down forp = 17 (although 
not under assumption of the generalised Riemann hypothesis) . This is blissfully 
irrelevant for the equation = z^, since Kraus’s result applies. 

3 The Equation 

The construction of the curves Cp^t does not depend on p > 5. In constructing 
the model + Ap^t we did use that p is odd, though, so we cannot use 

those models for p = 4. As it turns out, C 4 - 1 , € 4^0 and C 44 are genus 1 curves 
with a rational point. Therefore, they are elliptic curves over Q. Unfortunately, 
€ 4^-1 and € 4^0 have Mordell-Weil groups of rank 1, so they have infinitely many 
rational points. This is useless if we want to bound the number of primitive 
solutions to x^ + y^ = z^. For this case, we find other parametrising curves. 
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of higher genus. We use that fourth powers are squares. Thus, a solution to 
_|_ y3 _ ^4 jg g^^gQ ^ solution of = v^, with v = z^. We use Zagier’s 

result to describe those solutions. 

Lemma 1 (Zagier). Let z G Z be coprime integers satisfying x^ + y^ = z^ . 
Then there exist s,t G Z{2,3} with (s, t) ^ (0, 0) modp for any prime p \ 6 such 
that 

r a; or y = f a; or y = |(s^ + — 3t^) 

< y or X = — + 6s^t^ + 3t^ or < y or a; = |(— + 3t^) 

[ z = 6st{s'^ + [ z = jsti^s'^ + ?>t'^) 

{ X or y = s(s^ + 8t^) 
y or X = 4t(t^ — s^) 

±2 = s® - 20s®f® - 8t® 

See [2] . A detailed account of how to arrive at the parametrisations for a;^ + y^ = 
z® can be found in [3, Section 3.2]. The proof of Lemma 1 proceeds similarly. 

For a solution of a;® + y® = v"^ to be a solution of a;® + y® = z^ as well, we see 
that the z in Lemma 1 should be a square. So, by change of variables, we see 
that a primitive solution to a;® + y® = z^ gives rise to a rational point on one of 
the genus 2 curves 

Cl : = QX{X^ + 3) 

C2 : = 3A(A^ + 3) 

C3 : y2 = A® - 20A® - 8 
C4 : -Y^ = A® - 20A® - 8. 

So we can determine the primitive solutions of a;® + y® = z^ by finding those 
rational points and tracing them back to solutions of a;® + y® = z^. We hope 
that Chabauty methods apply to these curves and, as it turns out, things work 
out very well. 

Lemma 2. The Mordell-Weil groups of the Jacobians of the curves C\, C 2 , C3 
and C4 are finite. 

Proof: This can be showed by determining the size of jac(Ci)(Q)/2jac(Ci)(Q) by 
means of a 2-descent as described in, for instance [5]. This is quite a complicated 
procedure to carry out by hand but, fortunately, completely automated (see 
[21]). We will not bother the reader with boring details. □ 

Therefore, it is sufficient to determine the torsion part of the Mordell-Weil 
groups. For that, we use a trick that often works and is computationally very 
easy. Consider a curve C of genus g over Qp, given by a smooth projective model 
over Zp. Suppose that the reduction of that model, Cmodp, is again a smooth 
curve over Fp. Let ff be the Jacobian of C. Then reduction modulo p induces a 
splitting of J (Qp) 

0 ^ ^ Ji.%) ^ (f^modp)(Fp) ^ 0. 

The kernel of reduction, denoted by is a Zp-module. Consequently, 

it has only p-power torsion. By a more involved argument ([18, Theorem IV. 6. 4] 




On Powers as Sums of Two Cubes 



175 



or [5, Theorem 7.4.1]), it follows that the kernel of reduction is actually free of 
torsion. We will only use that all torsion of l7(Qp) prime to p maps injectively 
to {J modp)(Fp). Since 17(Q) injects in f7(Qp) for allp, we have for any pair of 
primes p, q of good reduction that 



I gcd(p’'#(f7modp)(Fp),g^#(f7modg)(Fg)) for some r, s. 

If C is given by = F{X), where F is some squarefree polynomial over Zp, 
where p is an odd prime not dividing the discriminant of F, then C can be given 
by a smooth model with smooth reduction, so the same principle holds (see [5, 
Theorem 7.4.1] for an even stronger result). 

In order to count points on Jacobians, we first have to represent them. We 
briefly review some standard results that can be found in [5] . Let C be a genus 
2 curve over a field K and let be its Jacobian. A point in J{K) can be 
represented by a divisor of C over K - i.e., a formal linear combination of points 
on C. Suppose we either have one Weierstrass point oo € C{K) or two points 
oo"'",oo“ rational over K or conjugate quadratic over K that are interchanged 
by the hyperelliptic involution on C. Then we can represent each divisor class by 
[P+Q — 2oo] or [P+Q — oo+ — oo“], where P,Q & C{K) or quadratic conjugate 
over K. This representation is even unique (apart from interchanging P and Q) 
for all divisor classes apart from the trivial one. The trivial divisor class, the 
neutral element of J{K), is represented by any divisor that counts the zeros of 
a function on C with multiplicity, where poles are counted as zeros with negative 
multiplicity. 

Lemma 3. Let C he a genus 2 curve over a finite field F^ and let J be its 
Jacobian. Then 

J(F,) = i(#C(F,))2 + l#C(F,.)-9 

Proof: Some simple combinatorics using the fact that divisor classes are either 
represented by a pair of rational points or a pair of quadratic conjugate points 
prove this fact. Alternatively, evaluate the characteristic polynomial of the Frobe- 
nius endomorphism at 1. See [5, Section 8.2]. □ 

For z = 1, 2, 3, 4, we write fJi for the Jacobian of C,. 

Lemma 4. J'i(Q) = J 2 (Q) = {0, [(0,0) -oo]}, so Ci(Q) = C 2 (Q) = {(0,0), oo}. 

Proof: The two divisor classes given, are clearly defined over Q. We find using 
Lemma 3 that #(j7mod 5)(Fs) = 26 and #(j7mod7)(F7) = 64 for both Ja- 
cobians. Since we have already seen that the Mordell-Weil groups consist solely 
torsion (Lemma 2), we see that (Q) ] gcd(5’’ • 26, 7^ • 64) = 2, which concludes 
the proof. □ 

Lemma 5. J74(Q) := {0, [(1 J- -\/3, 0) J- (1 — -\/3, 0) — oo+ — oo“]}, so C 4 (Q) = 0. 



Proof: We find #(f74 mod 5)(Fs) = 36 and #(j74mod 19)(Fig) = 484. Therefore, 
#J 4 (Q) I 4. Put Fi = -2X - 2, F 2 = X-^ + 2A3 -b - 4A -b 4. Then 
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— 20X^ — 8 = FiF 2 - Furthermore, the extension of Q generated by a root 
of F2 is Q(-\/3, V^)- A localisation of this field at a prime not above p = 2,3 
is at most a quadratic extension of Qp. Therefore, we have X\,X2, roots of F2, 
that are either rational or quadratic conjugate over Qp. In reduction, we have 
[(Si, 0) + {x 2 , 0) — 00 + — oo“] G (f 74 modp)(Fp). Note that two times this divisor 
is the divisor of the function {X — x\){X — X 2 ), so it represents a 2-torsion point 
on (f 74 modp)(Fp). So, we see that f7(Q) C Z/2Z x Z/2Z. 

Using -[(a;i,|/i)-|-(a; 2 ,y 2 )-oo+-oo“] = [(xi, -j/i) -I- (a; 2 , -j/ 2 ) - 00 + - oo“], 
we see that a divisor represents a point of order 2 only if j/i = j /2 = 0. As we 
have already seen, there is only one rational divisor class with that property. □ 

Note. The above proof shows that it is impossible to bound the torsion of 7/4 (Q) 
purely by local data at primes of good reduction, since f 74 has extra 2 -torsion 
at all good primes. An alternative proof would be the following. Upon closer 
inspection, (f 74 mod 5 )(F 5 ) has the structure Z/ 6 Z x Z/ 6 Z. Therefore, f 74 (Q) is 
either Z/2Z or Z/2Z x Z/2Z. From the data used to prove Lemma 2, we know 
that MQ)/2J4{Q) ~ Z/2Z. 

Incidentally, this curve is a nice example of the fact that local means are not 
always sufficient to determine torsion. This adds importance to the height theory 
on genus 2 curves, which does give an effective procedure, as is described in [5, 
page 82] and [22]). For determining C 4 (Q), however, it is entirely unnecessary to 
consider f 74 (Q). It is straightforward to check that C 4 (Q 2 ) = C 4 (Q 3 ) = 0- 

Lemma 6 . J 3 (Q) = ([(1 -k v^, 0) -k (1 - v^, 0) — oo’*' — 00 ],[oo“'' — 00 ]) and 
#f 73 (Q) = 6 - Furthermore, C 3 (Q) = {oo+,oo“}. 

Proof: Note that the divisor of the rational function y— a;^-kl0 is ±(3oo“'' — 3oo“). 
Therefore, 3[oo+ — oo“] = 0. 

This proves that the points mentioned in the lemma generate a group of 6 
elements. It remains to show that there are no other points. Upon inspection, 
we find #(J 3 mod 5 )(F 5 ) = #(J 3 mod 7 )(F 7 ) = 36, and the group structures are 
Z/ 6 Z X Z/ 6 Z. Using the same argument as in Lemma 5, we find that f 73 (Q) has 
only one point of order 2. This means that either f 73 (Q) ~ Z/ 6 Z or f 73 (Q) ~ 
Z/ 6 Z X Z/3Z. 

First we prove that C(Q) has no affine point. Suppose P = (x, y) ^ { 00 +, 00 “} 
with = a;®— 20a;^— 8. If P G C 3 (Q), then [P— 00 +] G J 3 (Q), so 0 = 6 [P— 00 +] = 
[ 6 P— 3oo''‘ — 3oo“] — 3[oo''‘ — oo“] = 3[2P— oo"*" — 00 “]. We construct a function 
g that has (at least) a quadruple zero in P and has triple poles in 00 “'" and 
00 “ . As it turns out, this fixes the location of the other two zeros. Therefore, if 
6 [P — oo’*'] = 0, then g has a zero of order 6 in P. 

Put F{X) = ^ 6 - 20^3 -8 and let g{X,Y) = Y - {g^X^ + g2X^ + giX + go) 
be a function on C 3 with a quadruple zero at P and a triple pole at both oo’*' and 
00 “ . Since F{X) has no rational roots, without loss of generality, y^ = F{x) yf 0, 
so t = X — a; is a uniformising coordinate at P. We compute a power series 
expansion of y in t at P. 

Op ■- oP -k O^pt ■■■ + + 0 (t®) = g{x + t, y^F{x + t)). 
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Since g{P) = 0, we have go = y — gsx^ — g 2 X^ — gix. We solve gi,g 2 , go from the 
equations 6*p^ = 9p'^ = = 0, which must hold for P to be a quadruple zero 

of g. We find 

1 ooo^^(^ ~ 1 )^( 3 ^ + 2)^(a;^ — 2x + 4)^(a;^ + a; + 1)^ 

“ " ’ 

648x(x — l)(a; + 2)(x^ — 2x + 4)(x^ + a; + l)(x^ + 2) 

(4a;9 - 111a;® - 168a;® + 32)/F(a;)4. 

Thus, we see that the only P with 6[P — oo+J = 0 and X{P) G Q are (0, 2V— 2), 
(l,3-\/^) and (— 2,6-\/6), and none of these points has Y{P) G Q. Therefore, 
C 3 (Q) = {oo+,oo“}. 

It follows that any 3-torsion point in f73(Q) other than ±[oo“'' — oo“], is of the 
form [(a;i, yi) + (x 2 , 2 / 2 ) — 00 + — oo“], where (a;i, yi) and (x 2 , 2 / 2 ) are quadratic 
conjugate points over Q and yi ^ 0. We use the same construction to show 
that such points do not exist. We determine go,. go such that the function 
Y — goX^ — g2X'^ — g\X — go has double zeros in {xi,y\) and (x2,y2)- Then 
we determine for which point the function has in fact zeros of order 3 in those 
points. One can do this by computing the conditions a;i -I- X 2 and a;ia; 2 . The 
computer algebra involved in this computation, is too bulky to display here, but 
completely straightforward. There turns out to be no other 3-torsion. □ 

Note. The proof of Lemma 6 exhibits that 6[(0, 2\/X2) — 00 +] = 6[(1, 3-\/^) — 
oo®'] = 6[(— 2,6-\/6) — oo+J = 0. For any prime p ^ 2, this yields an extra 
6-torsion point over Qp. This gives another example of a curve for which the 
torsion of the Jacobian cannot be determined solely by information at primes of 
good reduction. 

Proof of Theorem 1: Lemmas 4 through 6 give us the rational points on 
the curves C\ through C 4 . These points all correspond to trivial solutions of 
2-3 _|_ 2^3 _ ^4^ Thus, by construction of the curves, the only primitive solutions 
are trivial. □ 




4 The Equation 

As was shown in Section 2, it is sufficient to determine the rational points on 
the curves 

Cl : = X® - 559872 

C 2 : = X® - 62208 

C 3 : = A® - 6912 

in order to find all primitive solutions to x^ + y^ = 2 ®. Let Ji be the Jacobian of 
Ci. As a result of Theorem 3, we have seen that rk(j7i(Q)) < 1, rk(j 72 (Q)) < 1 
and that rk(j 73 (Q)) = 0. The procedure of determining the rational points on 
curves of genus 2 is, provided that generators of the appropriate Mordell-Weil 
groups can be found, almost a standard one, although not guaranteed to be 
successful. Instead of providing a lot of hard to check numerical data to make 
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up the proof, we will describe a session with the available software to give an 
idea how one can perform these computations in practice nowadays. 

Lemma 7. Ci(Q) = { 00 } 

We will prove this lemma using Chabauty techniques as described in [5], [10] 
and [4]. In these articles, the curve is embedded in the Jacobian using the map 
P I— > [2P — oo’*' — oo“] or [2P — 2ooj. We describe a session with the software 
mentioned in [4]. 

> read'divcalc.mpl' ; 

> initcurve (x~5+A) ; 

'current curve is y~2 = x~5+A 

> s_nG : =locexp( [n*L [1] ,n*L [2] ] ) : 

> k_nG:=loccoord2kuimner(s_nG) : 

> theta_nG : =f actor (series (kummer2thet a (k_nG) ,n, 11) ) ; 

theta_nG := series ( (4*L [2] ~5*(A*L[2] ~5+L[l] ~5) ) *n"10+0(n"12) ,n,12) 

> coef f _theta: =f actor (coef f (theta_nG,n, 10) ) ; 
coeff_theta := 4*L [2] (A*L [2] ~5+L [1] ~5) 

Let G G J7(Q) be a point in the kernel of reduction mod p of the Jacobian of a 
curve + A. The two quantities Li and L 2 can be computed from G and 

if the quantity coef f _theta does not vanish mod then nG is not of the form 
[2P — 2oo] for any n yf 0. Below, G will be a point in J7(Q). It is a generator 
of the Mordell-Weil group, but we will only need and check that it generates a 
group of finite index not divisible by certain primes. 

> alias (alpha=RootOf (x~2+3) ) ; 

I , alpha 

> F:=x~5-559872; 

F := x~5-559872 

> P : = [12*alpha, 1296+864*alpha] ; 

P := [12*alpha, 1296+864*alpha] 

> G:=[P,conj (P)] ; 

G := [[12*alpha, 1296+864*alpha] , [-12*alpha, 1296-864*alpha] ] 

> initcurve (F) ; 

'current curve is ', y~2 = x~5-559872 

> njac(ll,F) ; 

131 

> njac(19,F) ; 

400 

The last two instructions count the number of points on the Jacobian in reduc- 
tion. The values are coprime, so there is no rational torsion. 

> alphainl9:= (Roots (op (alpha) )mod 19); 
alphaml9 := [[15, 1], [4, 1]] 

> Gredl9 : =subs (alpha=alphainl9 [1] [1] ,G)mod 19: 

> tbll9:=maketablep(19,F mod 19, [[]] ,Gredl9) : 

> rowdim(tbll9) ; 
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20 

> select (i->tbll9[i+l, 2] [1] [1] =tbll9 [i+1 , 2] [2] [1] , 

[i$i=l . . rowdim(tbll9) -1] ) ; 

[8, 12] 

> G7red29:=adp(29,mlp(29,3,G) ,mlp(29,4,G)) ; 

G7red29 := [[21, 27], [25, 15]] 

> tbl29:=maketablep(29,F mod 29, [[]] ,G7red29) : 

> rowdim(tbl29) ; 

30 

> select (i->tbl [i+1 ,2] [1] [1] =tbl [i+1 , 2] [2] [1] , 

[i$i=l . . rowdim(tbl29) -1] ) ; 

[] 

> alpham7:=(Roots(op(alpha))mod 7); 
alpham? := [[2, 1], [5, 1]] 

> Gred7 : =subs (alpha=alpliam7 [1] [1] ,G)mod 7: 

> njac(7,F) ; 

50 

> mlp(7, 10,Gred7) ;mlp(7,25,Gred7) ; 

[[0,4], [0,4]] 

[[3, 0], [infinity, 0]] 

This shows that 20G € and that any divisor nG = [2P — 2oo] has 

n = 20m or n = 8 + 20m or n = 12 + 20m for some m G Z. However, the 
reduction data at 29 shows that any such point has n = 30m for some m G Z, 
so this rules out the last two options. We see that G generates the full group 
{J mod7)(F5) of order 50. Therefore, the group generated by G has index prime 
to 10 in the Mordell-Weil group. 

> Gml20:=ml(20,G) : 

> sGml20 : =div21occoord(Gml20) : 

> lGml20 : =loclog(sGml20)mod 19~3; 
lGml20 := [4636, 2660] 

> subs ( A=-559872 , L [1] =lGml20 [1] /19 , L [2] =lGml20 [2] /19 , 

coef f _theta)mod 19; 

14 

Here, we compute 20G and its 19-adic logarithm. The fact that [4636, 2660] 

[0, 0] mod 19^, proves that 19 \ [17(Q) : (G)j. Furthermore, we check that indeed 
the coefficient of in 6 does not vanish mod 19^^. This proves that OG is the 
only multiple of 20G of the form [2P — 2ooj. See [4] for details on how this power 
series argument works. 

Lemma 8. C 2 (Q) = {oo}. 

Although in principle the same procedure applies to this curve as well, we will 
use another method, described in [3]. An advantage of this method is that it 
might still work if rk(f7(Q)) > 1. Put a® = 2. We use that ii x,y € Q with 
— 62208, then there is a (5 in some finite set and j/i, j /2 G 71 = Q(a) such 

that 

X — 6a^ = Syi 

+ Oa^x^ + 72ax + 432a^x + 5184a^ = 
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We can therefore suffice in finding the it'-rational points on genus 1 curves with 
an ^-coordinate in Q. It turns out we can suffice in proving 

Lemma 9. The K -rational points on the genus 1 curve -h 6a^X^ -\- 

72aX 432a^X -|- 5184a^ with rational X -coordinate have X G {oo, 0, —12}. 

Lemma 10. The K -rational points on the genus 1 curve (1 -I- 2a — 2a^ -I- 2a^ — 
2a^)L^ = X'^ -\- 6a^X^ -\- 72aX -\- 432a^X -|- 5184a^ with rational X-coordinate 
have X G {12}. 

Note that all finite ^-coordinates found satisfy X^ < 62208, so they do not 
correspond to rational point on € 2 - Therefore, C 2 (Q) = { 00 }. More details on 
how to do this can be found in [3]. Here, we describe a session with a package 
that does these computations for you. 

kash> Read (" ell. g" ) ; 
ell package loaded. 
kash> pol : =x~5-62208 ; 
x~5 - 62208 

kash> 0 : =DrderMaximal (x~5-2) ; 

Generating polynomial: x~5 - 2 
Discriminant: 50000 
kash> OrderClassGroup(O) ; 

[ 1 , [ 1 ] ] 

kash> alpha : =X0rderPrimElt (0) ; 

[0, 1, 0, 0, 0] 

kash> theta :=PolyRoots(pol+RingZero(D)) [1] ; 

[0, 0, 0, 6, 0] 
kash> Qpol : =x-theta; 

X + [0, 0, 0, -6, 0] 
kash> Rpol : =pol/Qpol ; 

x~4+ [0 , 0 , 0 , 6 , 0] *x~3+ [0 ,72, 0 , 0 ,0] *x~2+ [0,0, 0,0,432] *x+ 
[0,0,5184,0,0] 

kash> deltas : =FilterTwists (Qpol , Rpol) ; 

[ 1 , [ 1 , 2 , - 2 , 2 , - 2 ] ] 

The last result shows that indeed we only need consider two twists of the genus 1 
curve (FilterTwists checks which twists have points with the desired property 
locally) . 

kash> ec:=Quar(x~3-5*x~2+Elt(D,5)*x) ; 

kash> EllAddHint (ec , 1) ; EllAddHint (ec , l-2*alpha+alpha~3) ; 
kash> EllGensMod2 (ec) ; 

Finding generators of 2-isogeny selmer group on curve... 



Found global basis using hints. 

[(0:0:1),([1,-2,0,1,0] : [3, 0,-2, -2,1] :1), (1:1:1)] 
kash> EllGenInit ( [EllXtoPnt (ec, 1) , 

> EllXtoPnt (ec , l-2*alpha+alpha~3) , EllXtoPnt (ec, 0)] ,2); 
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The function Quar initialises an elliptic curve in Weierstrass form from a model 
of the form = F{X). Since in this call, F is cubic, transforming to Weierstrass 
form is trivial. The next commands give hints to the system about where to look 
for ^-coordinates of Mordell-Weil generators. The command EllGensMod2 tries 
to find generators of E{K) /2E{K). First it bounds the rank using a 2-descent 
or a 2-isogeny descent (see [18]), depending on the curve. Then it makes a feeble 
attempt at finding generators. This is where giving hints helps tremendously. We 
register the found points. We will only prove that certain primes do not divide 
the index of the generated group in the full Mordell-Weil group. Note that we 
already know that the index is odd. 

kash> cov : =QuarCov (Rpol , ec) ; ; 

kash> pl51 : =PlaceSupport (151*0) ; ; 

kash> List (pl51 ,p->EllGrpIndex(ec mod p)); 

[ 2 , 2 , 2 , 1 , 1 ] 

This shows that the images of the group we have determined in the reductions 
at the several places over 151 (this is the smallest completely split prime. It is 
not necessary to use a split prime, but it does reduce the amount of needed 
computations) are at most 2. Since we know the index to be odd from the 
descent, we know that the image surjects on the reduction of the Mordell-Weil 
group. Finally, we check which points P have a rational ^-coordinate on the 
original quartic model. Note that for such a point, the values of X{P) under all 
5 embeddings K Qisi should agree. This gives 4 equations. Since the Mordell- 
Weil rank is 2, there are essentially only two variables, so we expect only finitely 
many solutions. See [3] for details. 

kash> EllCovChabCcov, [0 , [-12 ,1] , [1,0]] ,pl51) ; 

Result of FibStrict : [151 , [ [139, 1] , [0 , 1] , [1 , 0] ] ] 

Computing Theta~G for G=( 4: 2: 1 ) . . . 

G is only point in fiber if matrix has maximal rank mod 151 
[ 9 60] 

[132 113] 

[ 1 115] 

[ 73 101] 

Computing Theta~G for G=( [4,-2,-3,0,2] : [2, -9, -7, 4, 7] : 1) . . . 

G is only point in fiber if matrix has maximal rank mod 151 
[ 44 55] 

[145 147] 

[140 2] 

[ 12 17] 

Computing Theta~G for G=( 0: 1: 0 ) . . . 

Point maps to infinity. Taking 1/phi 

G is only point in fiber if matrix has maximal rank mod 151 
[ 37 56] 

[106 79] 

[100 66] 
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[ 48 28] 

[[ 0 , 1 ], [ - 12 , 1 ] , [ 1 , 0 ]] 

kash> 

For the other elliptic curve, we proceed similarly. 

kash> ec:=Quar((x~3-5*x~2+Elt(0,5)*x)/deltas[2] ) ; 

Elliptic curve [1 , 2 , -2 , 2 , -2] *y~2=x~3-5*x"2+5*x over order 
generated by x~5 - 2 

kash> cov : =QuarCov (Rpol/ deltas [2] , 12 , ec) ; ; 

kash> p3 : =Placelnit ( (-l+alpha-alpha~2+alpha"3-alpha"4) *0) ; 

place [3, [1, 2, 1, 2, 1] ] above 3 

kash> EllAddHint(ec,Elt(D, [8,6,5,4,3] )) ; 

kash> EllAddHint(E1121so(ec) ,Elt(D, [109, 128, 112 , 80 , 52] ) /9) ; 
kash> EllGensMod2 (ec) ; 

Finding generators of 2-isogeny selmer group on curve... 
Computing 2-isogeny selmer rank. 



Found global basis using hints. 

[(0:0:1) , ([8, 6, 5,4,3] : [76,67,58,50,43] :1) , 

( [14308 , 8384 , 9136 , 4952 , 4324] /2601 : [-4461238 , -3728612 , -3260650 , 
-2970062 , -2572342] /132651 : 1) ] 

kash> EllGenlnit( [EllXtoPnt(ec,Elt(D, [8,6,5,4,3] )) , 

> EllXtoPnt(ec,Elt(0, [15, -20, 5, -10, 10])/ 12), 

> EllXtoPnt(ec,0)] ,2) ; 

kash> List (pl51 ,p->EllGrplndex(ec mod p)); 

[ 1 , 2 , 2 , 1 , 8 ] 

kash> EllCovChab(cov, [12] ,pl51) ; 

Result of FibStrict: [ 151, [ [12, 1 ] ] ] 

Computing Theta~G for G=( 0: 1: 0 ) . . . 

G is only point in fiber if matrix has maximal rank mod 151 
[148 37] 

[ 31 98] 

[113 95] 

[ 41 76] 

[ [ 12 , 1 ] ] 

kash> 



Lemma 11. C3 = {00}. 

Proof: Since we already know that the Jacobian has rank 0, we only need to deter- 
mine the torsion. The fact that (J 3 mod 7 )(F 7 ) = 43 and (J 3 mod ll)(Fn) = 375 
shows that there is none. □ 
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5 Availability of Programs 

Two implementations of the descent procedure on genus 2 curves as described 
in [5] are available, both written by Stoll. One is based on the public domain 
packages PARI/GP ([1]) and CLISP, the other is a package for MAGMA. The 
latter also has routines for applying Ghabauty methods on Jacobians of genus 2 
curves. The PARI/GP based program is available in binary form for Linux/i386 
systems from 

http : //www.math.uni-duesseldorf . de/~stoll/genus2/ 
and the MAGMA package can be obtained from 

http : //www.math.uni-duesseldorf . de/~stoll/programs/HC/. 

Routines for doing computations on Jacobians of genus 2 curves are available 
from several locations. There are some routines referred to in [5] at 
ftp://ftp.liv.ac.uk/pub/genus2 for Maple V. The author has also made 
available some routines for the commercial computer algebra package Maple V 
for doing computations. An example session is described in Section 4. In the 
same location, there are also some rudimentary routines for doing 2-descents, 
written for KASH ([7]). 

An elliptic curve package for elliptic curves over arbitrary number fields based 
on KASH is also available. It can do 2-descent and 2-isogeny descent on elliptic 
curves over number fields, also with even class number. Furthermore, as demon- 
strated, there are facilities for Ghabauty-arguments. 

The computations needed to determine the 3-torsion in Section 3, can be 
found in tor334.mpl. The proof of Lemma 7 can be found in prf335.mpl, 
but the Maple program in divcalc . sh, available from the same location, is 
also necessary. The elliptic curve Ghabauty method can be found in prf335.g, 
together with the KASH package ell . sh which it is based on. 

Electronic locations have a very temporary nature. All these files are presently 
located at http://www.math.uu.nl/people/Bruin/, but this will probably not 
be permanent. The author will attempt to have a home page somewhere with 
links to the relevant files. 
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Abstract. We give an efficient algorithm for factoring polynomials over 
finite algebraic extensions of the p-adic numbers. This algorithm uses 
ideas of Chistov’s random polynomial-time algorithm, and is suitable for 
practical implementation. 



1 Introduction 

Factoring polynomials over the p-adic numbers Qp is an important problem in 
computational number theory. One application is determining the prime ideals of 
a number field Q(a), and how a given rational prime p factors into prime ideals 
in that field. See Cohen [10] and the references cited therein for some methods 
currently in use. 

These algorithms, while generally good in practice, will take exponential time 
for some polynomials. A. L. Chistov ([7], [8], and [9]) has given an algorithm 
which runs in random polynomial time for all polynomials, but would be very 
difficult to implement efficiently. In this paper we give a random polynomial- 
time algorithm which works well in practice. The algorithm is non-deterministic 
only because all known efficient algorithms for factoring polynomials over finite 
fields Fpn ([3], [5]) are non-deterministic. Note that any polynomial-time p-adic 
factoring algorithm can factor polynomials over Fpn in polynomial time. It has 
been implemented in PARI, and is available on the second author’s web site [13]. 

We will factor polynomials over a finite algebraic extension K of Qp. See 
Chapter 5 of [14] for properties of these extensions. Let tt be a uniformizer of 
K. In the case when K is an unramified extension of Qp, we choose tt = p. For 
X in the ring of integers Ok of K, x will denote the image of x in the residue 
class field K. We will fix a set of representatives A = {0,oi, . . .Op/_i} C Ok 
for the elements of K. This set may be lifted to representatives for unramified 
extensions of A in a straightforward manner. 

The valuation of an element x G K will be denoted |a;|, and its order by ord x. 
We assume that | | has been normalized so that |p| = 1/p. There is a unique 
extension of the valuation | | on AT to its algebraic closure K ; we assume that | | 
has been so extended. 

Just as for real numbers, one cannot, in general, explicitly represent a p-adic 
number exactly, but only an approximation which is a rational number. Thus 
our algorithm will find approximations to factors of F{X). Elements x oi K may 



W. Bosnia (Ed.): ANTS-IV, LNCS 1838, pp. 185-208, 2000. 
© Springer- Verlag Berlin Heidelberg 2000 




186 



David G. Cantor and Daniel M. Gordon 



be written x = with Ui G A. In Section 8 we discuss where this 

series can be truncated to guarantee a correct answer. 

Let F(X) be a monic polynomial with coefficients in Ok which has no re- 
peated factors. See Zippel [29, pp 294-295] for a simple method of removing 
repeated factors. Unlike Chistov’s algorithm, our method does not require com- 
puting in, or even constructing, ramified extensions of K. The algorithm is ap- 
plied recursively, at each step either finding a new factor or terminating with an 
irreducible factor and certificate of its irreducibility. The certificate of irreducibil- 
ity will be a generalized Eisenstein polynomial with coefficients in the maximal 
unramified (over K) subfield of K{x), where a: is a root of the irreducible factor. 

The p-adic Factor algorithm works by looking for a polynomial A{X) for 
which we can determine the factorization of 

i?(F) = Resx(F(X),y-^(X)). (1.1) 

In Section 2 we show that a factor of R{Y) lets us find a factor of F{X), and a 
certificate of irreducibility for R{Y) also applies to F{X). Once such an A{X) is 
found, we apply the information to F{X) and, if necessary, recurse on remaining 
factors of the original polynomial. 

The standard “easy” method for factoring a polynomial over the p-adics, the 
Newton diagram method, is given in Section 3. If the Newton diagram of the 
polynomial is not a straight line, then Hensel’s Lemma may be used to find a 
factor. If the Newton diagram is a straight line with slope k/n, where n is the 
degree of F{X) and k is relatively prime to n, then F{X) is irreducible. 

Otherwise the Newton diagram method fails, and we use an extension of 
Hensel’s Lemma given in Section 4.1. We proceed by looking at the factorization 
of F{X) in K. If the reduction F*{X) (defined in Section 3) has two relatively 
prime factors, then using Hensel’s Lemma we may lift these to factors over K. 
If F*{X) is the power of an irreducible polynomial of degree d > 2, then we 
may factor F(X) over an unramified extension of degree d of K, leading to 
a factorization of F{X) over K. These methods form the basis of the Hensel 
Factor routine given in Section 4.2. The only case Hensel Factor cannot handle 
is when 

R{Y) = a„ (F” — 67 t^)™ -I- [terms above the Newton diagram]. (1-2) 

In this case we have ordH(a;) = s/r for each root x of F{X) in K, the closure of 
K. The p-adic Factor algorithm then finds a new polynomial A{X) such that 
either Hensel Factor successfully factors R{Y), or (1.2) still holds with either 
ord A{x) or deg A{X) increased. Since deg A{X) < n, and ord A{x) is bounded 
by Corollary 5.8, this will terminate after a bounded number of steps. 

In Section 7 we illustrate how the algorithm works on two examples. Section 8 
gives a worst-case bound for the bit complexity of the algorithm 

0{n^+Aog^\Ap\\oip'^), (1.3) 

where n is the degree of F{X), Ap is the discriminant of F{X), and k is the 
degree of K over Qp. 
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Our algorithm may be extended to any local field complete with respect to 
a discrete rank-1 valuation, under the assumptions that the residue class field is 
perfect and that an algorithm for factoring polynomials defined over the residue- 
class field is given. For example, applying it to the field Fg((Ai)) of Laurent series, 
it can be used to resolve singularities of plane curves. A future paper will extend 
the algorithm to other local fields, and include some proofs which have been 
omitted here due to space constraints. 

We thank Stephen DiPippo and Robert Segal for many helpful discussions. 
John Cannon told us of developments with MAGMA’s local rings and fields pack- 
age, and informed us that the MAGMA group has developed a similar algorithm 
for factoring polynomials over Qp, which is currently being implemented. 

2 Some Criteria for Factorization 

In this section we give simple criteria for polynomial factorization and polynomial 
irreducibility. Let Resx(A(A), i?(A)) denote the resultant of two polynomials 
A{X) and B{X). See Lang [19] or Cassels [6] for details. Due to space constraints 
we omit proofs of the lemmas in this section. They follow in a straightforward 
way from the properties of the resultant. 

Lemma 2.1. Suppose that F{X) and A{X) are polynomials in the held K[X] 
with F{X) monic of degree n. Put 

R{Y) = Resx{F{X),Y-A{X)). (2.2) 



Then 

1. R{Y) is a monic polynomial of degree n in Y and 

2. the polynomial F{X) divides the polynomial i?(A(A)). 

The following lemma provides a way of factoring a polynomial. 

Lemma 2.3. Suppose that F{X) and A{X) are polynomials in K[X], with 
F{X) monic. Put 

R(Y) = Resx(F(X),Y-A(X)). (2.4) 

Suppose further that R{Y) = Ri{Y)R 2 {Y) is a factorization of R{Y) into rela- 
tively prime, non-constant factors. Then 

F(A) =Fi(A)F2(A), (2.5) 

where 

Fi(A) = gcd(F(A),Ri(A(A)) and F^iX) = gcd(F(A), i? 2 (A(A)), (2.6) 

is a factorization of F{X) into relatively prime, non-constant factors. Further- 
more, 



degFi(X) = degRi(Y) and deg F 2 (A) = deg i? 2 (L). 



(2.7) 
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The following Lemma provides a partial converse to Lemma 2.3. 

Lemma 2.8. Suppose that F{X) is a monic polynomial of degree n, that A{X) 
is a polynomial, and that both have coefficients in the Geld K. If the polyno- 
mial R{Y) = Resx(T’(-^), L — is irreducible over K, then F{X) is also 

irreducible over K. 

If neither Lemma 2.3 nor Lemma 2.8 applies, we may need to go to an 
unramified extension field of K. The following lemma shows how irreducible 
factors of F{X) over an extension field L of K lead to irreducible factors over K. 



Lemma 2.9. Suppose that F{X) is a monic polynomial in K[X] with no re- 
peated factors of degree > 1, that L is a Gnite algebraic extension of K, and that 
G{X) is a monic, irreducible, polynomial in L[X] of degree > 1 which divides 
F(X). Put H{X) = XormL / kG{X). Then, 

1. gcd{F (X) , H (X)) is an irreducible factor of degree > 1 of F{X) in K[X]; 
and 

2. if the Geld extension Lj K is generated by the coefGcients of G{X), then 
H{X) is already an irreducible factor of F(X) in K[X], 

3 Newton Diagrams 

In this section we give our notation for Newton diagrams and some related items. 
For details see Artin [1], Cassels [6], or Gouvea [14, Section 6.4]. 

Suppose that 

n 

R{Y)=J2arY^ (3.1) 

i=0 

is a polynomial in K[Y] of (exact) degree n > 1. As usual, we associate to 
R{Y) a finite, non-empty point set © C consisting of points (z, ord a,) S 
corresponding to each nonzero term UiY^ of R{Y). 

Definition 3.2. We define, as is customary, the Newton diagram of R{Y) to be 
the lower boundary of the convex hull of 6. 

Following Cassels [6], we use the following definition: 

Definition 3.3. Suppose that R{Y) is a given by (3.1). We shall call R{Y) pure 
if oo yf 0, n > 1, and the Newton diagram of R{Y) is a straight line. 

If the Newton diagram is not pure, we may immediately factor R{Y). The follow- 
ing is is well known (see Cassels [6]), and is also a corollary of our Theorem 4.21. 

Lemma 3.4. Suppose that R{Y) = ^ polynomial of degree k > 1 

and that uq is not zero. If the polynomial R{Y) is not pure (so that its Newton 
diagram consists of two or more straight line-segments necessarily of different 
slopes), then R(Y) factors into two non-constant polynomials in K[Y], 
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If the Newton diagram is pure, we may sometimes use its slope to show that 
R{Y) is irreducible. 

Lemma 3.5. (Generalized Eisenstein criterion) Suppose R{Y) is pure, and its 
Newton diagram has slope k/n, where k is an integer relatively prime to n. 
Then R{Y) is irreducible. 

Proof. If y is a root of R{y) in K, then ordy = k/n. Hence K{y)/K is a totally 
ramified extension and has degree n, so R{Y) is irreducible. □ 



Remark 3.6. The customary form of Eisenstein’s criterion is the special case 
when k = —1 (see, for example, [29]). 

Now suppose that R{Y) is pure and has slope — s/r. Because the points 
(0, ordoo) and (n, orda„) are the end-points of the Newton diagram, n must be 
an integral multiple of r, say, n = mr. Put 

at = (3.7) 

so that G Ok- We can then write 

m 

R(Y) = a„ ^ -|- [terms above the Newton diagram]. (3.8) 

i=0 

Here “terms above the Newton diagram” refers to those non-zero terms of R{Y) 
whose corresponding points in the Newton set © lie strictly above the Newton 
diagram. These are the non-zero terms of the form aiY^ for which ord at > 
s{m — i)/r + ord a„. 

Definition 3.9. Suppose R{Y) as given by (3.1) is pure and suppose that the 
oii are given by (3.7). Define 



R*{Y) = ^dciY\ (3.10) 

i=0 

The polynomial R*(Y) is monic and has coefficients in K. In the next section 
we will show how to factor F{X) using Hensel’s Lemma if we can write R*{Y) 
as the product of two relatively prime factors, perhaps over an extension field 
of K. Otherwise, we will use a reduction method extending the one used by 
Chistov [8]. 

4 Factoring with Hensel’s Lemma 

4.1 Hensel’s Lemma 

Hensel’s Lemma refers to an algorithm, due to Hensel [17], which shows how 
to find a factorization of a polynomial R(Y) G K[Y] from an “approximate 
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factorization”. Here we describe an extension of this algorithm. The extension is 
related to that of Artin [1]. The main novelty is Corollary 4.30. In the special 
case when the slope of the Newton diagram of R{Y) is zero, it is well known. 
Dealing with general slopes avoids the need to go to ramified extension fields as 
in [8], making the algorithm much more practical. 

Definition 4.1. Suppose that A is a positive real number. If 

k 

A{Y) = Y,a^Y^ ^K[Y]. (4.2) 

i=0 

define its X-norm ||A(y)||A to be max^ |ai|Ab If A is understood we shall write 
simply ||A(y)|| instead of ||A(y)||A. 

When A{Y) is the constant polynomial oq, that is, when n = 0, then ||A(y)||A 
= |ao|, independent of A. Suppose A = then, llaA^HA = |a7r^|. If A(V) = 

is pure (see definition 3.3) with slope —s/r then ||A(y)||A = |ao|. 

Lemma 4.3. Suppose that 

1. A{Y) = ^ polynomial in K[Y] of degree k; 

2. B{Y) = non-zero polynomial in K[Y] of degree I < k; 

3. \\B{Y)\\ = ||6/y'||; equivalently, |6/|A' = max^ \bi\\\ 

Define C{Y) = A{Y) — Y^~\ak/bi)B{Y). In other words, C{Y) is the first 
remainder and {ak/bi)Y^~^ is the first quotient obtained when dividing A{Y) by 
B{Y) using the classical division algorithm. Then 

1. ||C'(y)|| < ||A(y)||, and 

2. \\{ak/k)Y'^-^\\<\\A{Y)\\/\\B{Y)\\. 

Proof. Define bi = 0 when i < 0. Then 

k 

C{Y) = ^ (au-i - Y<^-\ (4.4) 

i—1 



Hence, 



||C'(y|| = max A'^-* 

l<i<k 



^k — i 



o-kbi-i 

bi 



< max max 

l<i<k 




Afe|gfc|A^->;-iK 
A'|6z| ) 



< max max (A^ *|afc_i|, A^|afc|) 

0<Z<fc 



(4.5) 



= PWII- 



The remainder of the proof is clear. 



□ 
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Lemma 4.6. Suppose that A{Y) and B{Y) are polynomials satisfying hypoth- 
esis 1, 2, and 3 of Lemma 4.3. Suppose that Q{Y) and V{Y) are the quotient 
and remainder, respectively, when A(Y) is divided by B(Y); that is, 

A{Y) = B{Y)Q{Y) + V{Y), (4.7) 

where A(Y), B{Y), Q(Y'), andV(Y) are polynomials in K[Y] such that degV (Y) 
< deg b\y). Then 

||y(y)|| < P(y)|! and ||g(r)|! < p(y)||/||i?(y)||. (4.8) 

Proof. Apply Lemma 4.3 repeatedly. □ 



Lemma 4.9. Suppose that we are given a 7-tuple 

{k, ^r, B{Y),C{Y),u{Y),v{Y),e{Y)) (4.10) 

where k is a positive integer, where p, is real number > 1, and where the remain- 
ing five entries are polynomials in Suppose that the following conditions 

are satisfied: 

1. B{Y) = C'(L) = non-zero polynomials in K[Y] 

of degrees, respectively, I and m, such that 

lii?(y)|| = WkY^w = ||C'(y)|| = 1; (4.11) 

2. ||u(r)|| < p and ||f(L)|! < p; 

3. \\u{Y)B{Y)+v{Y)C{Y)-l\\ < 1; 

4. deg e{Y) < k and I -\-m < k. 

Then there exist a pair of polynomials {U{Y),V{Y)), each in such 

that: 

1- ||C^(i^)|| <Ml|e(L)|| anddegU{Y) <k-l; 

2. ||V"(L)|| < p\\e{Y)\\ and degy(L) < ^ - 1; 

3. \\U{Y)B{Y) + V{Y)C{Y) - e(r)|| < ||e(y)||. 

Proof. From hypothesis 3 we obtain 

\\e{Y)u{Y)B{Y) + e{Y)v{Y)C{Y) - e(r)|| < ||e(y)|| (4.12) 

Let Q{Y) be the quotient and V{Y) be the remainder when e(L)u(L) is 
divided by B{Y)\ that is, e(L)u(L) = Q(Y)B(Y) + V(Y), where Q(Y) and 
V{Y) are polynomials in K\Y] with deg V{Y) < ^ — 1. By Lemma 4.6, 



linmi < ||e(L)u(y)|| <Ml|e(nil 



(4.13) 
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and 



||g(y)|| < ||e(y)r;(y)||/||i?(y)|| <Ml|eWII 

Next, 

e{Y)u{Y)B{Y) + e{Y)v{Y)C{Y) - e{Y) 

=e{Y)u{Y)B{Y) + {Q{Y)B{Y) + V{Y))C{Y) - e{Y) 
= {e{Y)u{Y) + Q{Y)C{Y))B{Y) + V{Y)C{Y) - e{Y) 
=U'{Y)B{Y) + V{Y)C{Y) - e(F), 

where 



Then, 



and 



U\Y) = e{Y)u{Y) + Q{Y)C{Y). 
\\U'{Y)\\ < max(||e(y)u(y)||, ||Q(y)c(y)||) 

<A<y)\\ 

\\U'{Y)B{Y) + V{Y)C{Y) - e(y)|| < ||e(r)||. 



( 4 . 14 ) 



( 4 . 15 ) 



( 4 . 16 ) 



( 4 . 17 ) 



( 4 . 18 ) 



The polynomial V (Y) already meets the requirements of the Lemma. We show 
that we can modify U'{Y) to obtain the required polynomial U{Y). Write 

U'{Y) = J2urY\ ( 4 . 19 ) 



If any monomial UiT* satisfies ||uiT*|| < ||e(T)||, then we may replace Ui by 0; 
this will not affect the validity of (4.18). Define U{Y) to be the polynomial 
obtained from U'{Y) by replacing all such monomials UiY'^ by 0. Then, 

\\U{Y)B{Y) + V{Y)C{Y) - e(y)|| < ||e(y)||. (4.20) 

Put j = deg U{Y). If j < k — I, we are done. If not, then, the term of highest 
degree in the product U{Y)B{Y) has degree j+l > k. Since degV{Y)C{Y) < I — 
1+m < k and deg e{Y) < k, the term of highest degree in the product U (Y)B{Y) 
must also be the term of highest degree in the left-hand side of U{Y)B{Y) + 
V{Y)C(Y) — e(P). The norm of this term is |jujP-’|| ||6/P^|| > ||e(P)||. This 
contradicts (4.20) and shows that j + I < k, equivalently deg C/(P) < k — 1. □ 

For the remainder of this section we assume that A is a rational power of |7t|. 
Specifically, A = |7 t|^/’’, where r and s are relatively prime integers with r > 1. 
In particular, we require that if s = 0, then r = 1. Under this assumption, the 
norm ||^(y)|j of any non-zero polynomial A(Y) € K[Y] will be an integral power 
of |7t|^/’’. 

We can now state the form of Hensel’s Lemma that we use. 
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Theorem 4.21. (Hensel’s Lemma) Suppose that h is a non-negative integer 
and that we are given a 5-tuple of polynomials 

(R(Y),Bo(Y),Co(Y),u(Y),v(Y)) (4.22) 

each with coefficients in K such that 

1. R(Y) has degree k and satisfies ||i?(T)|| = 1; 

2. Bq{Y) = degree I and satisfies ||i?o(L)|| = ||6/T^|| = 1; 

3. Co{Y) = YlT=o^i^'' degree m and satisfies 11(70(^)11 = 1; 

4. ||i?(y) -i?o(L)Co(r)|| < |7 t|(2'«+i)A; 

5. ||zr(y)|| < |7 t|-'*A, ||„(y)|| < |^|-Vr. 

6. ||u(y)Bo(T) + v{Y)Co{Y) - 111 < 1. 

Then there exist polynomials B(Y) and C{Y) in K[Y] such that 

1. R{Y) = B{Y)C{Y); 

2. ||i?(y)-i3o(L)|| < Itt^A; 

3. ||C(y)-Co(r)||<|7r|V- 

4. degB{Y)=degBo{Y). 

Proof. We first show that we may assume that k > m -\- 1. If k < I -\- m, then 
the term of highest degree of R{Y) — Bo{Y)Co{Y) is —biCmY™-^^ whose norm, 
by hypotheses (2) and (4), satisfies 

||6;y'|| ||c„y™|| = ||6;c^y'+™|| < IttI^^'^+da, ( 4 . 23 ) 

so that ||cmL™|| < |7 t|(^^+^^/’’. It follows that if we replace Cq{Y) by by the 
lower degree polynomial CofY) — CmY™ and replace m by the degree of this new 
Cq{Y), then the hypotheses remain satisfied. For the remainder of this proof we 
assume that k > I -\- m. 

We shall construct sequences of polynomials {Bi{Y)} and {Ci{Y)} for i = 
1,2,... such that 

1. \\B,{Y) - B,_i{Y)\\ < \ 7 T\(>^+d/r and deg B,{Y) = 1; 

2. \\Ci{Y) - a_i(T)|| < |7 t|('*+*)/’' and dega(T) < m - 1; 

3. \\R{Y) - B,{Y)Ci{Y)\\ < |7T|(2'*+i+i)A, 

Putting B(Y) = limi^oo .Bi(y) and C(Y) = limj^oo C'i(T) will complete the 
proof. 

We proceed by induction on the variable i, starting with i = 1. Put €i(Y) = 
R{Y) — Bi-i{Y)Ci-i{Y) so that, by hypothesis (when z = 1) or induction (when 
z > 1), ||ei(T)|| < |7 t|(^^+*)/’’. Apply Lemma 4.9 to the 7-tuple 

(fc, |7^|-^ B,{Y),C,{Y),u{Y),v{Y),e,{Y)). (4.24) 



Lemma 4.9 returns a pair of polynomials which we denote {Ui{Y), Vi{U)). These 
polynomials satisfy 
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1. ||J7i(F)|| < and deg Ui{Y) < m — 1; 

2. ||V"i(i^)|| < |7t| and degyi(F) < ^ - 1; 

3. \\U,{Y)Bo{Y) + V,{Y)Co{Y) - e,(r)|| < , 

Define 



B,{Y) = B,_i{Y) + V,{Y), C,{Y) = Ci_i(F) + Ui{Y) (4.25) 



Then 

\\R{Y) - B,{Y)C^\\ 

= ||i?(y) - {B,-i{Y) + y,(y)) (a_i(r) + u.{y))\\ 

= \\{R{Y) - B,_i{Y)C,_i{Y)) 

- {Ui{Y)B,_i{Y) + Vi{Y)C,.i{Y)) - U,{Y)Vi{Y) 

= ||(e,(y) - {u,{Y)B,_i{Y) + y,(y)C'i_i(y))) 

-i7,(y)y,(y)|| 

<max(|7rp'*+i,|7rp'*+2*) 

_ |^|(2?i+i+l)/r 



□ 

The proof of Hensel’s Lemma consists of an algorithm. If only approximations 
to the factors R{Y) and B{Y) are needed, then the algorithm is finite. We shall 
call the algorithm Hensel’s Lemma, also. 

Now suppose that we are given a polynomial R{Y) which is pure and whose 
Newton diagram has slope —s/r, where r and s are relatively prime integers 
with r > 0. The degree of R{Y) must be a multiple of r, say kr. Both of the 
points (0,ordoo) and (fcr, ordofcr) must lie on this segment. We can write 

k 

R{Y) = Y,a. 7T + [terms above the Newton diagram] (4.27) 

i=0 

where jaij < 1 for 0 < i < k, and where, In the A = jTrj^/’’ norm, 

||i?(y)|| = jaol = = Wk\- (4.28) 

Equation (4.27) can be restated as 

k 

||i?(y)-^a.7r— wni < ||i?(y)||. (4.29) 

i=0 



When this is the case we have 

Corollary 4.30. Suppose that R{Y) is a pure polynomial of degree kr, of 
the form (4.27) which satisfies (4.28) and suppose further that the polynomial 
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R*{Y) = satisfies R*{Y) = (}{Y)^{Y) where P{Y) and ^{Y) are 

monic, relatively prime polynomials in Then R{Y) = B{Y)C{Y) where 

B{Y) and C{Y) are relatively prime polynomials in K[Y] satisfying B*(Y) = 
/3(Y) and C*(Y) = j(Y). 

Proof. By multiplying R(Y) by an appropriate power of tt, we may assume 
that |ji?(F)|| = 1. Suppose that deg /3(F) = I and deg 7(F) = m. There exist 
polynomials /i(F) and v(Y) in K[Y] such that ^(F)/3(F) + v(Y)y(Y) = 1 and 
such that deg^(F) < m and degi^(F) < 1. Choose elements bi, Ci, Ui, and Vi in 
K such that 



i 

/3(F) = ^6.F\ 

m— 1 

^i{Y) = ^ u,Y\ 

Define 

i 

Bo{Y) = J2brn-^"Y", 

i^O 
m— 1 

u{Y) = ^ u^tt-^^Y", 

i—0 i=0 

Then Bo{Y)* = /3(F), Co(F)* = 7(F), u{Y)* = ^i(F) and u(F)* = ^i(F). Apply 
Theorem 4.21 with ft, = 0 to the 5-tuple 

(i?(F), i3o(F), Co(F), u(F), u(F)). (4.33) 

The result will be two polynomials 33(F) and C{Y) which meet the requirements 
of this corollary. □ 

The special case of this Corollary when C{Y) is pure with horizontal Newton 
diagram appears as Lemma 4.1 in [6]. 

4.2 Hensel Factor 

We may now define Hensel Factor, an important subroutine of our algorithm. 
It takes as input a triple {K, F{X), A{X)), where K is a field, F{X) is a poly- 
nomial of degree > 2 to be factored, and A{X) is a non-zero polynomial of 
degree < degF(A). We will say the algorithm succeeds if one of Lemmas 3.4, 
3.5 or Corollary 4.30 apply. If Lemma 3.5 holds, then {K , F (X) , A{X)) forms 
a certificate for the irreducibility of F{X), and we are done. If Lemma 3.4 or 
Corollary 4.30 hold, then we have found a factor G{X) of F{X) over a field 
L, and we recursively call p-adic Factor with input (L,G{X)). If none of the 
lemmas apply, we say it fails. 



7(F) 

/-I 

KF)=^u.FL 



(4.31) 



z=0 



Co(F)=^c,7^-*W*^ 

z^O 

/-I 

v{Y)='^ViTT-^^Y". 



(4.32) 
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Hensel Factor. Input {K , F (X) , A{X)) . 

1. Compute R{Y) = Resx {F{X),Y - A{X)). 

Comment. Each of the elements A{x), where a; is a root of F{X), is a root 
of R{Y). If the resultant R{Y) were a monomial, then the n distinct roots x 
of F{X) would satisfy the polynomial A{X), of degree < n. Thus R{Y) is 
not a monomial. 

2. There are now four sub-cases, at most one of which can hold: 

(a) The polynomial R{Y) is not pure. 

Factor R{Y) using Lemma 3.4. Then factor F{X) using Lemma 2.3. Let 
G{X) be a factor of least degree. Restart p-adic Factor with the pair 
{K,G{X)). 

(b) The polynomial R{Y) is pure and R*(Y) can be written as a product of 
two relatively prime factors, each of degree > 1 in K[X], 

Factor R{Y) using Corollary 4.30 of Hensel’s Lemma. Then factor F{X) 
using Lemma 2.3. Let G{X) be a factor of least degree. Restart p-adic 
Factor with the pair {K,G{X)). 

(c) The polynomial R{Y) is pure and R* (F) is the e**' power of an irreducible 
monic polynomial a{Y) of degree > 2 in R'[F]. 

Choose a polynomial u{Y) € K[Y] such that u{Y) = a{Y). Denote by L 
the unramified extension field of K obtained by adjoining a root y of 
u{Y) to K. Put (}{Y) = [Y — yY and put 7 (F) = R*{Y)/ (}{Y). Then 
R*(Y) = f3{Y)j{Y) where (/3(F), 7 (F)) = 1. By Corollary 4.30 we can 
factor R{Y) as R{Y) = B{Y)G{Y) where B*{Y) = /3(F). Factor F{X) 
over L using Lemma 2.3 with Ri{Y) = 33(F) and R^iX) = C(F). Let 
Fi{X) be the factor of F{X) corresponding to i?i(F). Restart p-adic 
Factor with the pair {L, Fi{X)). 

Comment. Note that the field L is determined uniquely by K and 
a(F); it is independent of the specific choice of u{Y) (see Artin [1, page 
69, Theorem 2A]). Moreover, if a; is a root of Fi{X) in K, then y = Fi{x). 
Hence the field L is contained in the field K{x). 

(d) The polynomial R{Y) is pure and the slope of its Newton diagram is 
k/n where (fc, n) = 1 . 

By Lemma 3.5, F{X) is irreducible and the algorithm terminates with 
the triple {K, F{X), A{X)). 

3. None of the four cases (2a), (2b), (2c), or (2d) applies, so that R*{Y) is a 
power of a linear factor in 3f[F]. 



Return failure 
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5 Some Technical Lemmas 

We state here some simple results which will be used in the next section. We 
first have a lemma from elementary number theory. Its proof is constructive. 

Lemma 5.1. Suppose that h is a positive integer and that for I < j < h we 
are given fractions sj /r^ where Vj and Sj are relatively prime positive integers. 
Define to = 1 ^.nd for 1 < j < h, define tj = lcm(ri, r 2 , . . . ,rj). Then, for any 
integer u, there exist integers ej, for 1 < j < ti, satisfying 0 < ej < tj/tj-i and 
such that 

h 

(5.2) 

i=i 

is an integer. 

Proof. The proof proceeds by induction on h. When h = 1, then ti = ri, and 
the unique choice for ci is the least non-negative, integral solution to eiSi = 
u (mod ri). 

Suppose that h > 1. We will show that there exist integers v and Ch such 
that 0 < eh < th/th-i and such that 

ehShIrh + vlth-i -ufth (5.3) 

is an integer. This will reduce the problem to the h — 1 case with u replaced 
by V. Multiplying (5.3) by th shows that we must choose eh and v to satisfy 

ehShthfch + vthith-i = u (mod U) (5.4) 

Now suppose that p is a prime dividing th, that p“||r^ (this means that p“ is 
the exact power of p dividing rh), and that p^\\th-i- Put 7 = max(a,/3). Since 
th = \cm{th-i,rh), we see that p~*\\th. Then p~*~°'\\{th/rh) and p~<-l^\\{th/th-i). 
If a = 7 , then p divides hence does not divide Sh, so that p does not divide 
Shth/ch. If /3 = 7 , then p does not divide th/th-i- Thus p divides at most one of 
Shth/ch and th/th-i. It follows that Shth/ch and th/th-i are relatively prime. 
Hence there exists a solution eh and v to (5.4) (even with equality replacing 
congruence). For any integer k the pair (eh + kth/th-i,v — kshth/ch) is also a 
solution of (5.4). Replacing eh by eh + kth/th-i for an appropriate integer k 
allows us to choose eh to satisfy 0 < eh < th/th-i- □ 

This immediately gives the following corollary, which will be used in the 
algorithm to construct a polynomial E(X) with specified values of E(x) for the 
roots X of E(X). 

Corollary 5.5. Suppose that h, the fractions Sj/rj and the integers tj satisfy 
the hypotheses of Lemma 5.1. Suppose that Hi, H 2 , ... ,Ah are elements of Qp 
such that ord Aj = sj / rj . Then for any integer u there exist integers e\, 02 ,. . . ,Ch 
satisfying 0 < ej < tj/tj-i and an integer cq such that ord7r®“ 0^=1 ^j^ = 




198 



David G. Cantor and Daniel M. Gordon 



The next lemma shows that if a monic polynomial of degree m is “small” at 
n > m distinct points, then at least two of these points must be “close” to each 
other. If the points are given in advance, then there is a limit to how “small” 
the polynomial can be at all n points. 

Lemma 5.6. Suppose that xi,X 2 ,-- - ,Xn are elements of K and that A{X) 
is a monic polynomial in of degree m < n. Then minjyji \xj — < 

maxi |^(xi)|. 



Proof. Put e = maxj \A{xj)\. We can write A{X) = Y\^^i{X — 9i) where the 
9i G Qp are the roots of A{X). Then for each j, 

m 

e>\A{xj)\=Y[\xj-9i\. (5.7) 

Not all of the factors \xj—9i\ on the right-hand side of (5.7) can be > Hence 
there must exist a value of i, call it cr(j), such that \xj — 6*^0)! < By doing 
this for all j, we obtain a map a from the set {1, 2, . . . , n} to the set {1, 2, ... , m}. 
Since n > m, there must be two values, j yf f such that a(j) = a{f). Call this 
common value k. Then both la;,- — aA < and \xp — aA < Hence 

\xj-Xj^\<e^/^ □ 



Corollary 5.8. Suppose that F{X) is a monic polynomial in Ok[X] of degree n 
with distinct roots xi,X 2 , ■ ■ ■ ,Xn- If A{X) is a monic polynomial in of 

degree m < n, then, for at least one i, we have |^(a;i)| > \ Ap\^. 

Proof. Because all |a;i| < 1, we have 

Ap =\\\xi — Xj\ <min\xi — Xj\ (5-9) 

1'G3 

Now apply Lemma 5.6. □ 



6 The p-Adic Factor Algorithm 

In this section, we describe the main algorithm. It will find an irreducible factor 
H{X) of F{X) along with a certificate that H{X) is irreducible. To completely 
factor F{X), the algorithm may have to be repeated, perhaps several times, with 
F{X) / H{X) replacing F{X) until this quotient is 1. 

The algorithm will attempt to factor F{X) using Hensel Factor with A{X) 
= X. This will fail only when F*{X) has the form {X — a)'". When this occurs, 
the algorithm will systematically look for a polynomial A{X) G K[X] for which 
Hensel Factor succeeds. 

Because the algorithm is recursive and both the polynomial to be factored 
and the local field may change during the course of the algorithm we will, for the 
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remainder of this paper, denote by Fq(X) the original polynomial to be factored 
over the original field Kq. 

The input to the algorithm is a pair (K,F{X)), where K is either Kq or a 
finite, unramified extension of Kq, and F{X) is a monic polynomial of degree 
n > 2 with coefficients in Ok dividing Fo{X). We assume F{X) has no multiple 
factors and F{0) yf 0. Since we compute approximations to the factors, F{X) will 
not in general be known exactly. In Section 8 we determine how much precision 
is needed to avoid errors in the factorization. 

The p-adic Factor algorithm will return a field L which is an unramified 
extension of K of degree < n, a polynomial G{X) in L[X] dividing F{X), and 
a polynomial B{X) G L[X] of degree < degG(Ar). By Lemma 3.5, the triple 
(L,G{X), B{X)) provides the proof that G{X) is irreducible. 

By Lemma 2.9, 



H{X) = XoimL/K G{X). (6.1) 

is an irreducible factor of F{X). As noted above, the algorithm may then be 
called recursively on the pair {K, F{X) / H{X)) to complete the factorization of 
F{X). ^ 

Section 6.1 presents the algorithm, after which Section 6.2 describes in more 
detail what certain steps are doing, and why they work. 

6.1 The Algorithm 

p-adic Factor. Input: {K, F{X)). 

Step 1. Apply Hensel Factor to (K,F(X),X) (in this case 
Resx{F{X),Y-X) = F{Y)). 

Step 2. We reach this step only if Hensel Factor did not succeed in Step 1, so 
F*{X) is a power of a linear polynomial. Choose a G A such that 

F{X) = (A’’ — OTT^)™ + [terms above the Newton diagram] (6.2) 

where 

(a) d is the unique root of F*(A) in A and orda = 0; 

(b) r < n and m > 1; 

(c) mr = n; gcd(r, s) = 1; 

(d) the Newton diagram of F{X) has slope —s/r. 

Step 3. We initiate the outer loop by putting Ai{X) = X, R\{Y) = F(Y), 
ri = r, Si = s, to = 1, and G = ri. 

Step 4. (Outer loop) For h = 1, 2, . . ., perform Steps 5 through 11. 

Step 5. To begin the inner loop, put 

Bo(X) = Ah(xy’‘/^>‘~\ 

So(Y) = Resx(F(X),Y - Bo(X)), 

^0 — ^ht}i / ij'hth—l) • 
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Step 6. (Inner Loop) For z = 0, 1, . . perform Steps 7 through 10. 

Step 7. Use Corollary 5.5 to choose integers ej, for 0 < j < h, such that 

(a) 0 < Cj < tj/tj-i — 1 when 1 < j < h, 

(b) Co + = Ui/tfi (in the notation of Corollary 5.5, cq = 

Define a polynomial E{X) by 

E{X) = Tr^°Ai{xy^A2{xy^---Ah{xy’'. (6.3) 

Step 8. Define 

C{X) = Bi{X) E{X)-^ (mod E{X)) (6.4) 

and 

r(U) = Resx(F(X),y-C(X)). (6.5) 

Apply Hensel Factor to the triple {K, E{X), C{X)). 

Step 9. Put B{X) = Bi{X)-aE{X) and S{Y) = Resx{F{X),Y-B{X). Apply 
Hensel Factor to the triple {K, E{X), B{X)). 

Step 10. If the common value ordi?(a;) can be written in the form u/th, where u 
is an integer, then put Bi+i{X) = B{X), S'j+i(F) = S{Y), m+i = u. 
and continue the “inner loop” by returning to Step 6. 

Step 11. Denote the common value of ordi?(a:) by Sh+i/^h+i, where Vh+i and 
szi+i are relatively prime, non-negative integers as before. Put Ah+i (A) 
= B{X), Rh+i{Y) = S{Y), and th+i = lcm(t,,, r^+i). 

(a) If th+i < n continue the “outer loop” by returning to Step 4, with 
h increased by 1. 

(b) Otherwise use Corollary 5.5 to choose integers Cj for 0 < j < ft- -I- 1 
such that 

i. 0 < €j < tj/tj-i — 1 when 1 < j < ft and 
ii- ejSj/rj - l/th+i = eo; 

Define E{X) by 

E{X) = n^°Ai{xy^A 2 {xy^ ■ ■■Ahixy’' ( 6 . 6 ) 

and apply Hensel Factor to the triple {K, E{X), E{X)). 

6.2 Discussion of the Algorithm 

In Step 2, each (unknown) root x of E{X) has ordx = s/r by (6.2). This shows 
that the ramification index of each of the n field extensions of the form K{x) I K 
is divisible by r. 

Starting with Ai{X) = A at Step 3, the outer loop defines a finite sequence 
of polynomials Ai(A), A 2 (A), . . . and a corresponding sequence of pairs of non- 
negative integers, (ri, si), (r 2 , S 2 ), . . ., where each of the pairs (rj, s,) are rela- 
tively prime. We have Rh{Y) = Resx(F(A), F — A;,(A)), to = I, and for ft > 0, 
define th = lcm(ri, V 2 , ■ ■ ■ , ru)- The the following properties are easily checked: 
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1. Each of the and each of the th divides n. 

2. The polynomial Ah{X) is monic of degree t^-i- 

3. There exists an element A such that order = 0 and 

Rh{Y) = (y’’'* — + [terms above the Newton diagram]. 

It follows that for each root x of F{X), we have 

OTdAh{x) = Sh/rh. (6.7) 

Thus the multiplicative group generated by [tt], |Ai(x)|, \A 2 {x)\, ■ ■ ■ , \Ah{x)\ 
is independent of the choice of x and contains the value group of K. Hence, 
for each root x of F{x), the ramification index of the field extension K{x) j K 
is divisible by r/,. 

4. The integer does not divide th-i and for each root x of F(X), the rami- 
fication index of of the field extension K{x)/K is divisible by th- It follows 
that t\ < t 2 < ■ • • < th < n. 

Since ti is a proper divisor of ti+i, we must have h < log 2 n. This limits the 
number of steps of the outer loop. 

To determine Ah+i{X), we attempt in the inner loop to find a monic poly- 
nomial B{X) of degree th satisfied by all roots x of F{X). Since F{X) has 
n > th distinct roots, this attempt must fail. Its failure either leads to a situation 
where we can factor F{X) using Hensel’s lemma or leads to the determination of 
Ah+i{X). The inner loop finds Ah+\{X) by defining a sequence of polynomials 

Ho(X),Hi(X),H2(X),... (6.8) 



and a corresponding, strictly increasing, finite sequence of non-negative integers 

Uo < Ui < U2, 

Each polynomial Bi(X) is monic of degree th- Each root x of F{X) will 
satisfy ordi?i(a;) = Ui/th- Corollary 5.8 provides an upper bound for Ui and 
hence the sequence Bo{X), Bi{X), . . . will be finite. 

In Step 7, we have constructed E{X) so that ordE(a;) = Ui/th for every 
root X of F{X). Since degHj(Ai) < tj-i, we obtain, from Step 7a, have 



degE{X) < '^{tj/tj-i - I)tj_i 
i=i 

h 

= -tj-i) 

i=i 

= th-l. 



(6.9) 



In Step 8, (6.4) is valid because E{X) and F{X) have no common zeros. The 
polynomial T(Y) is monic of degree n and, for each root x of F(X), we have 
[(7(0;) I = \Bi{x) / E{x)\ = 1. Consequently, the Newton diagram of T{Y) is the 
horizontal line-segment connecting the points (0,0) and (n, 0). It follows that 
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the polynomial T* (Y) is monic of degree n and its constant term is not zero. If 
Hensel Factor fails, then we can write 

T{Y) = {Y — a)” + [terms above the Newton diagram] (6.10) 

where a £ A and ord a = 0. 

After Step 10, since Bi{X) is monic of degree th and degA(A) < th, B{X) 
is monic of degree th- By the definition of a, we have ord Bi(x) — aE(x) > 0 
for each root x of F{X). It follows that ord i?(a;) > ordi?i(A) for each such x. 
If Hensel Factor fails, then ordi?(a;) is the same for all roots x of F{X) and 
is ^ Uilth- 

Put 6 = ord I Ap |. Step 6 will increase i by 1. Since th divides n and the 
the Ui are non-negative integers and strictly increasing we have Ui/th > ijn. By 
Corollary 5.8, we see that Ui/th < Sn. Thus i < SiA. This means that for each 
value of h, the inner-loop is performed at most SiA times. 

In Step 11a, Vh+i does not divide th, so that th+i > th- In Step lib, we have 
ordif(a;) = 1/n for every root x of F{X), so case 2d of Hensel Factor will 
succeed, and this will lead to finding an irreducible factor of Fq{X)- 

7 Two Examples 

We decided to implement the algorithm, both to verify its correctness and prac- 
ticality, and to allow experimentation. The first decision was to choose a mathe- 
matical package in which to implement it. MAGMA [4] was the original choice, 
but a package to perform local field operations was delayed several times, so the 
implementation was done in GP instead. GP is a part of the PARI system devel- 
oped by Henri Cohen [2] . It does support p-adic fields, and is flexible enough to 
support unramified extension fields of the p-adics relatively easily. A new version 
of MAGMA with local fields has recently appeared, so a port of the algorithm 
to MAGMA is planned. 

The resulting code is available at the second author’s web site [13]. Because 
of the overhead of GP, it is slower than the PARI routine f actorpadic for most 
polynomials. An implementation in C using the PARI library would run in about 
the same time as f actorpadic for most polynomials. 

For an example of how the algorithm functions, we will factor the polynomial 

F{X) = {X-4f{X'^ -2)+2^°° (7.1) 

over Q 2 - 

If we apply p-adic Factor to this polynomial, it starts by attempting to 
apply Hensel Factor. The Newton diagram of R{Y) = F{Y) is not pure, so 
using Hensel’s Lemma we find factors 

Gi(A) = (A^ -2) + (2^°i -h 2^°® + ---)X+ (2®® -h 2^°® + ■ ■ ■) (7.2) 

and 



G2{X) = {X- 4)® -h (2101 -h 2102 + . . . )x -p (2®o -h 2ioo + - - -) (7.3) 
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Attempting to factor Gi{X), we call Hensel Factor again. This time, the New- 
ton diagram is pure, and we are in subcase (2d). Thus Gi{X) is irreducible. 

G 2 {X) is also pure, but its slope and degree are both even, so Hensel Factor 
does not apply. We have G^iX) = (A — 1)^. 

In Step 2 of p-adic Factor, we have a = 1, r = 1, s = 2, and n = m = 2. 
We arrive in Step 7 with E{X) = 4, G{X) = A/4, and 

T{Y) = Y'^-2Y+ {1 + 2^^ + ■■■). (7.4) 

The Newton diagram of T(Y) is now horizontal, but T*(Y) = (Y — 1)^ is 
still a power of a linear polynomial, so the call to Hensel Factor in Step 8 fails. 
In Step 9, we have a = I and B{X) = A — 4. This gives 

S'(A) = (2i°i -h • • • ) + (2®® + •••)■ (7-5) 

The call to Hensel Factor in Step 9 now goes to subcase (2d), and we have 
proved that G 2 {X) is irreducible, completing the factorization of F{X). 

Very few polynomials make it all the way through the inner loop more than 
once. One that does is 

F(A) = (A^ -2-2^°)(A2 -2-h2^°) (7.6) 



over Q 2 - 

We have F*{X) = (A — 1)^, so Hensel Factor fails. In Step 2 we choose 
a = I, r = 2, s = I, m = 2, and n = I. Entering the inner loop, we find 
F{X) = 2, C(A) = AV2, and 

T{Y) = - 4V® -h (6 - 2®®) -h (-4 -h 2^®) V -h (1 - 2®® -h 2^®). (7.7) 

Again, Hensel Factor fails. In Step 9 we set B{X) = A^ — 2, and have 

S{Y) = - 24iy2 28®. (7.8) 

Hensel Factor fails on S{Y), and ordi?(a;) = 20 for each root x of F{X), so 
we continue the inner loop. Returning to Step 7, we have E{X) = 2^®, G{X) = 
2 ~ 2 ®a 2 _ and T(Y) = Y'^ — 2Y^ + 1. Once again, Hensel Factor fails. 

Finally, we succeed in Step 9. This time we have B{X) = X^ — 2 — 2^®, and 
S{Y) = -I- -I- 2^^V^. The factor of in S{Y) yields the factor 

Gi(A) = A^ -2- 2^®. (7.9) 

Both this factor and the other one immediately are shown to be irreducible 
by subcase (2d) of Hensel Factor. 

8 Bounds on Required Precision and Complexity 

From the discussion in Section 6.2, it is clear that the loops of p-adic Factor 
will be executed a polynomial number of times in n and log | Ap \ . Therefore, to 
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show that p-adic Factor is a random polynomial-time algorithm, we only need 
to bound the precision needed in the computations. 

In general, we can only approximately compute the factors of the p-adic 
polynomial F(X). This causes two problems. First, in the gcd computation in 
Lemma 2.3: 



F,{X) = gcd{F{X),R,{A{X)), (8.1) 

we do not know the Ri exactly, and so terms in the computation that appear to 
be zero may not be. In this situation it is difficult to give a reasonable a priori 
estimate of the accuracy of Ri{Y) that is needed to compute the gcd to the 
desired accuracy. 

To circumvent this difficulty, we give an alternative method of computing 
Fi{X), which involves solving a system of linear equations. 

Lemma 8.2. Suppose that F{X) G K[X] is a monic polynomial of degree 
n with distinct roots xi,X2, . . . , in the algebraic closure K of K. Suppose 
that A{X) G K[X], Put yt = A{xi), and suppose that the yi are distinct. Put 
R{Y) = Resx{F{X), Y—A{X)). Then there exists a polynomial B{X) G K[Y] of 
degree < n — 1 such that B(A(X)) = X (mod F{X)). Furthermore, if R(Y) = 
Ri(Y)R 2 (Y) is a nontrivial factorization of R(Y), then F{X) = Fi{X)F2{X) 
where Fi{X) = Resy(i?i(F), X — B{Y)). Finally, deg Fi{X) = deg Ri{Y). 

Proof. We first show that the n polynomials A{X)’^ (mod F){X) for 0 < k < 
n — 1 are linearly independent over K. Suppose that we have a relation 



n—1 

biA{xy = 0 (mod F{X)). (8.3) 

Substituting the values x = Xk into (8.3) yields the system of linear equations 

n—1 

biyl, = 0 for 1 < z < n. (8.4) 

i=0 

The matrix of the equations (8.4) is a Vandermonde. Since the yk are distinct it 
is nonsingular. This shows that all of the bi are zero. It follows that the equation 

n—1 

Y biA{Xy = X (mod F{X)) (8.5) 

has a unique solution bo, bi, . . . bn-i- Put B{Y) = Then 

n—1 

B{A{X)) = Y biA(Xy = X (mod F{X)). 

2=0 



( 8 . 6 ) 
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Suppose that deg = r. By renumbering we may suppose that the roots 

of Ri{Y) are ?/i, j/2, ■ ■ ■ ,yr where r < n. The roots x of Fi{X) are those x for 
which there exists y such that R{y) = 0 and x — B{y) = 0. Thus the roots of 
Fi{X) are a;i,a;2, ■ ■ ■ ,Xr where Xi = B{yi). This shows that Fi{X) is a factor 
of F{X) of degree r. Similarly, F2{X) is a factor of F{X) of degree n — r. It is 
immediate from the definition of resultant that degTi(A') = degi?i(A'). □ 

The other potential problem of using approximations to Ri{Y) is that, if 
we do not use sufficient accuracy, the factorization might be changed. Corol- 
laries 8.7 and 8.19 give bounds on the accuracy needed to preserve the correct 
factorization. 

Corollary 8.7. Suppose that R{Y), Bq{Y), and C'o(T) are polynomials in Y 
of degrees k, I, and m, respectively, and 



\\R{Y) - Bo{Y)Co{Y)\\ < |Resy(i?oW,Co(r))|". (8.8) 

Then if the polynomials R{Y), Bo{Y), and C'o(T) satisfy hypotheses 1, 2, 
and 3 of Hensel’s Lemma, there exist an integer h and polynomials u(Y) and 
v{Y) such that h and the 5-tuple [R{Y), Bo{Y),Co{Y),u{Y),v{Y)) satisfy the 
hypotheses and hence the conclusions of Hensel’s Lemma. 

Proof. Put 



h = r ■ ordRes(i?o(P), C'o(P)). 



(8.9) 



Then, using this value of h, hypothesis 4 of Hensel’s Lemma is satisfied. 

We will choose polynomials u{Y) and v{Y) in K[Y] of degrees < m — 1 and 
< ^ — 1, repectively, to satisfy 



u(T)Ro(n + i’(i")C^o(i") = 1. 



( 8 . 10 ) 



Suppose that u{Y) = Equation (8.10) 

amounts to a system of ^ -I- m linear equations in the I m unknowns, uq, 
ui, . . . , Um-i and vq, v\, . . ., u/_i. The matrix of this system of linear equations 
is, up to sign, the Sylvester (resultant) matrix of Bo{Y) and C'o(P) (see, for 
example, [10], Section 3.3.2). Since the determinant of this matrix is non-zero, 
the coefficients of u{Y) and v(Y) are uniquely determined elements of K, not 
all 0. We may estimate them as elements of the field K. Choose t G K to satisfy 



r = 7T 



so that jrj = [[tt = 1/A. Put 



Then, 



u{Y/t), 


Bl{Y)=Bo{Y/r), 


v{Y/t), 


Cl{Y) = Co(P/r). 


lk(E)||, 


|jRJ(E)||i= ||i?o(E)||, 


HY)\\, 


IIQ(E)||i = ||Co(r)||. 



( 8 . 11 ) 



( 8 . 12 ) 
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Substituting F/r for Y, equation (8.10) becomes 

u^Y)B^,(Y) + v^(Y)CUY) = 1 (8.13) 

As above, equation (8.13) may be considered as a system of linear equations in 
the coefficients of u'^(z) and v'^(z), which may be obtained from the matrix of 
equation (8.10) by elementary row operations, giving 



It follows that 



and similarly 



\u^/F\ < l/|Resy(i?o(P),C'o(r))| . 


(8.14) 


lk(P)|| < |Resy(i?o(P),Co(r))|-\ 


(8.15) 


lk(P)|| < |Resy(Ro(R),Co(P))|-^ 


(8.16) 



Thus the remaining hypotheses of Hensel’s Lemma hold. 



□ 



This corollary shows that if R{Y) is computed to accuracy given by (8.9), 
then any factorization found will be correct. To show that a proof of irreducibility 
is also not changed by small pertubations of R(Y), we first need two easy lemmas. 

Lemma 8.17. Suppose that B{Y) and C{Y) are polynomials in K\Y] whose 
product A{Y) is pure. Then both B{Y) and C{Y) are pure. Furthermore, the 
Newton diagrams of the three polynomials A{Y), B{Y), and C{Y) have the same 
slope. 



Proof. This follows by repeated applications of Theorem 3.1 and Lemma 3.2 of 
Chapter 6 of [6]. 



Lemma 8.18. Suppose that A{Y) and B{Y) are polynomials in K[Y] of the 
same degree k. Suppose further that ||A(y) — B{Y)\\ < ||A(y)|j. Then, if A{Y) 
is pure, so is B(Y) and their Newton diagrams have the same slope. 

Proof. Put a = ||A(y))||. Suppose that A{Y) = = 

Then \ai\ < a\~'‘ and |oi — hi\ < aA“L It follows that \hi\ < aA“L 
Since |ao| = |a| and |ofc| = |a|A“^, we see that |6o| = |a| and that \bk\ = |a|A“^. 



Corollary 8.19. Suppose that R{Y) is an irreducible polynomial of degree n 
satisfying ||i?(y)|| = 1, so that, in particular, R{Y) is pure. Suppose that the 
Newton diagram of R{Y) has slope —s/r < 0. If Ro{Y) is a polynomial of degree 
n satisfying ||i?o(i^)|| = 1 and ||i?o(i^) - ^(i^)|| < min(l, | I)^, then Rq{Y) 

is irreducible. 
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Proof. Suppose that -Ro(^) factors as Ro{Y) = Bo{Y)Co{Y). By Lemma 8.18, 
Ro{Y) is pure, and by Lemmas 8.17 both Bo{Y) and Co{Y) are pure, and 
their Newton diagrams have slope —sjr. We may assume that ||i?o(l^)|| = 
11(70(1^)11 = 1. Using the definitions and standard properties of the resultant 
and discriminant (see Lang [19, pp 200-204]), we find that | | = | |, and 

that I Res(i?o(U), (7o(U))| > Hence 



|jR(y)-Ro(U)|| < |Res(Ho(U),Co(U))|". (8.20) 



By Corollary 8.7, R{Y) factors, contradicting the hypotheses. □ 



Theorem 8.21. Let K be an extension of degree k of Qp, and F{X) G K[X] 
have degree n. Algorithm p-adic Factor will find an irreducible factor of F{X) 
in random time 



0(n8+nog3|ZiF|log"/). (8.22) 

Proof. By Corollaries 8.7 and 8.19, we will find the correct factorization if we 
compute terms to 0{\ Ap P) precision. Note that, although we are starting in 
an extension of degree k of Qp, we may need to go to an extension of degree n 
of that field. 

The dominant computation is the resultant, which in worst case takes time 
0(n^log^(| Ap pnp”^)) (see [10], Section 3.3). From the discussion in Section 6.2, 
the outer loop of the algorithm will be executed at most O(logn) times, and the 
inner loop at most 0(n^ log | Z\f |) times. When Hensel Factor succeeds, we 
may have to call p-adic Factor on a factor of degree at most n/2, so that no 
more than O(logn) recursive calls will be needed. Combining these bounds, we 
have (8.22). □ 

The implied constant in (8.22) depends upon the choice of uniformizer tt 
and representatives A. Note that this is a pessimistic worst-case bound. Most 
polynomials factor on the first call to Hensel Factor, and it takes an effort to 
construct a polynomial which goes through the inner and outer loops more than 
once. Since we have not used fast arithmetic algorithms, and it is unlikely that 
all the worst cases can occur simultaneously, with a more detailed analysis the 
n^+<^ in (8.22) can be improved. 
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Abstract. A critical step when factoring large integers by the Number 
Field Sieve [8] consists of finding dependencies in a huge sparse matrix 
over the field F 2 , using a Block Lanczos algorithm. Both size and weight 
(the number of non-zero elements) of the matrix critically affect the run- 
ning time of Block Lanczos. In order to keep size and weight small the 
relations coming out of the siever do not flow directly into the matrix, 
but are filtered first in order to reduce the matrix size. This paper dis- 
cusses several possible filter strategies and their use in the recent record 
factorizations of RSA-140, R211 and RSA-155. 



Introduction 

The Number Field Sieve (NFS) is the asymptotically fastest algorithm known 
for factoring large integers. It holds the records in factoring special numbers 
(R211 [4]) as well as general numbers (RSA-140 [3] and RSA-155 [5]). One dis- 
advantage is that it produces considerably larger matrices than other methods, 
such as the Quadratic Sieve [1]. Therefore it is more and more important to 
find ways to limit the matrix size. This can be achieved by using good sieving 
parameters and by “intelligent” filtering. 

In this paper we describe the extended version of the program filter which 
we implemented following ideas of Peter L. Montgomery. Its goal is to speed up 
Block Lanczos’s running time by reducing the matrix size but still keeping the 
weight under control. 

A previous implementation of the program filter [8, section 7] did 2- and 
3-way merges. When using Block Lanczos, higher-way merges were commonly 
banned from the filter step in order to limit the matrix weight. For instance, 
also James Cowie et al. [6, section Cycles] explicitly avoided merges higher than 
3 for the factorization of RSA-130. 

The most important new ingredients of the present filter implementation 
are an algorithm to discard excess relations and “controlled” higher-way merges. 
We determine arithmetically which merges reduce Block Lanczos’s running time. 

For the factorization of RSA-140 only 2- and 3- way merges were performed 
which led to a matrix of 4.7 million columns. With the present filter strategy we 
could have saved up to 33% of linear algebra time by reducing the size to 3.3 
million columns. For the factorization of R211 we already used an intermediate 
filter version which did 4- and 5- way merges, but we could still get an improved 
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matrix after the factorization. For RSA-155, we could take full advantage of the 
present version and did “controlled” merges up to prime ideal frequency 8 which 
led to a matrix of 6.7 million columns and an average of 62 entries per column 
which was used to factor the number. Afterwards, we were able to reduce this 
size to 6.3 million columns. 

First, we give a brief description of the NFS. Secondly, the filter imple- 
mentation will be described with special focus on the new features. In section 3 
we will describe other filter strategies we came across in the literature and com- 
pare it with our approach. Finally, experimental results for RSA-140, R211 and 
RSA-155 are listed and interpreted. 

1 Brief Description of NFS 

We briefly describe the NFS factoring method here, skipping parts which are 
not relevant for the understanding of this paper such as the sieving step itself. 

By N we denote the composite number we would like to factor. We select an 
integer M and two irreducible polynomials f{x) and g(x) G Z[x] with cont(/) = 
cont(g) = 1 and f ^ ±g such that f{M) = g{M) = 0 mod A^. By a, /3 G C we 
denote roots of f{x) and g{x), respectively. 

The goal is to construct a non-empty set S of co-prime integer pairs (a, &) 
for which both ri(a6)es(® ~ ri(a6)es(® ~ squares, say, 7^ G 

Z[a] and G Z[/?], respectively. Once we have found S, the two natural ring 
homomorphisms (j)i : Z[a] ZjNZ mapping a to M and (f >2 : Z[/3] ^ Z/A^Z 
mapping (3 to M as well, yield the congruence 

(/)i(7)^ = (/)i(7^) = {a - hM) = = 4>2{5Y rood N . 

(a,fc)GS 

which has the desired form mod N. By computing gcd(A — Y, N) we 

may And a divisor of N. The major obstruction in this series of congruences 
is that we need to And 7 G Q(a) from 7^ (and S from <5^, respectively). See 
Montgomery’s [15] or Phong Nguyen’s [17] papers for a description of their square 
root algorithms. 

How to And the set S? We write 

F{x, y) = /(x/y)/®s(/) ^nd G{x, y) = 5(a;/y)/®s(s) 

for the homogeneous form of f{x) and g{x), respectively. Consider a — baG Q(a) 
and a — h(3 G Q(/3). The minus sign is chosen in order to have 

-^Q(a)/o(a - 6a) = F(a, 6)/ci and NQ(^pyq{a - bf3) = G{a,b)/c 2 , 

where the Cj’s are the respective leading coefAcients of f{x) and g{x). 

After the sieving we are left with many pairs (a, 6) such that gcd(a, 6) = 1 
and both F{a, b) and G(a, b) are products of primes smaller than the large prime 
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bounds L\ and L^, respectively, which were chosen by the user before the sieving. 
The pairs (a, b) are commonly denoted as relations. A necessary condition for 

(a — ba) and (a — b(3) 

(a,fc)GS (a,fc)GS 

to be squares is that the norms 

A(Q(a)/Q n (a — 6a) j and Aq(^)/q I (a — 6/3) j 

\{a,b)es ) \{a,b)es ) 

are squares. Therefore we require S to have even cardinality and 

F{a, 6) and G(a, 6) 

(a,fc)GS (a,fc)GS 

to be squares. The condition is not sufficient because elements having the same 
norm may differ from each other (not only by units!). Let p be a prime divisor 
of F{a,b) = }{a/b)b‘^^^^f\ We distinguish two cases: 

— p I f{a/b). This means that a/6 = q mod p with 0 < g < p is a root of f{x) 
modulo p. In the sequel such a p is referred to as p, q. 

— p I 6. Since gcd(a, 6) = 1 it follows that p \ a and therefore p | ci. This 
can happen for a small set of primes only, since the leading coefficient is of 
limited size. These roots are called projective roots and denoted as p, oo. 

We will call the couples p, q, where q is allowed to be oo, prime ideals, since 
they are in bijective correspondence with the first degree prime ideals of the ring 
Z[a] n Z[a“^]. See [2, Section 12.6]. 

Consequently, we write 

|F(a,6)| = and |G(a, 6)| = 

P.9 P.9 

In order for O/a 6)eS O/a 6)eS squares in Q(a) and 

Q(/3), respectively, we require all the exponents in 

\F{a,b)\ = Y[p^S<^ii<^.b,p,g) JJ- |G(a^5)| = ]^pEs«2(a.6.p.9) 

(a,6)eS P.9 (a,6)eS P.9 

to be even. This condition can be stated in terms of the field F2 as well. We 
just think of a relation (a, 6) as a vector in F2 whose first entry is 1 (in order 
to control the parity of S) and the following entries are given by the exponents 
Cl (a, b,p, r) and 62(0, b,p, r) modulo 2. A 1 signals the occurrence of an uneven 
power of a prime ideal. The task of finding some suitable sets S translates now 
into finding dependencies modulo 2 between the columns of a matrix which is 
built up with the relation vectors given by the siever. We need to have enough 
relations to guarantee that the matrix provides enough dependencies. 
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Alas, not every dependency yields a set S such that ri(a6)es(® ~ 
ri(a squares, but we can make the method practical by producing 

several dependencies and doing quadratic character tests [2, Section 8]. 

The filter stage occurs between the sieving step and the linear algebra step 
of the NFS. It is a preliminary linear algebra process since it corresponds to 
dropping columns {pruning) and adding up columns modulo 2 {merging). 

2 Description of the New Filter Tasks 

We distinguish 19 merge levels: level 0 and 1 fall into pruning, level 2 through 
18 within merging. 

We shall say that a prime ideal p, q is (un)halanced in a relation (a, b) if it 
appears to an (un)even number in F{a,b) or G{a,b)*. We distinguish between 
prime ideals of norm below and above a user determined bound filtmin. Ac- 
cordingly, we speak about small and large prime ideals. We will denote prime 
ideals p, q by I. We write a relation r = r(a, b) as the collection of its unbalanced 
large prime ideals, r : Ii, I 2 , ■ ■ ■ , Ik- Merging means combining relations which 
have a common prime ideal in order to balance it. For example, if I appears 
only in ri : Iio = I, In, . . . , Iiki and r 2 : I 20 = I, hi, ■ ■ - , hk 2 , we can combine 
the two relations into ri -|-r 2 : In , . . . , hki, hi, ■ ■ ■ , hk^ with the result that I is 
balanced in ri -|-r 2 . More generally, a k-way merge is the procedure of combining 
k relations with a common prime ideal / into k — 1 relation pairs without I. By 
a relation-set we mean a single relation, or a collection of two or more relations 
generated by a merge. We do merges up to prime ideal frequency 18. The pa- 
rameter mergelevel I means that fc-way merges with k < I may be performed. 
The weight of a relation-set r, i.e., the number of unbalanced prime ideals in it, 
is denoted by w{r). 



2.1 Pruning 

As the verb “pruning” suggests, this part of the program removes unnecessary 
relations from the given data, that is duplicates and singletons and, if the user 
wants to, also excess relations. Duplicates are obviously superfluous and single- 
tons cannot be part of a winning set S since they contain a prime ideal which 
does not occur in any other relation and can subsequently not be combined to 
form a square. If the difference between the number of relations and the num- 
ber of large prime ideals outnumbers a user-chosen bound (keep), the clique 
algorithm selects relations to delete. 

mergelevel 0 only removes duplicates and can be used to merge several 
sieving outputs to a single file, possibly before sieving completes, mergelevel 1 

* In very rare cases {p divides the polynomial resultant) we can have the same p, q 
appearing in both F and G. Recall that they are not the same, since they correspond 
to ideals in different rings. We abstain from labeling the ideals accordingly, for the 
sake of simplicity. 
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will only be performed if the full set of relations is available and covers algorithms 
for the removal of duplicates, singletons and excess relations. 

Duplicates. First we want to eliminate duplicate relations. They may arise for 
various reasons. Most commonly they come from sieving jobs that were stopped 
and later restarted. In case of a line-by-line siever [ 8 , section 6 ] the resumed 
jobs start with the last b sieved by the previous job; this is the only way that 
duplicates arise. In case of a lattice siever [18] the job starts with the special 
prime ideal I sieved last, and will generate duplicates, or it can do so because a 
relation may contain, apart from its own special I, other prime ideals that are 
used as special prime ideals as well. The simultaneous use of line-by-line and 
lattice siever also causes overlap. 

Duplicates are tracked down by hashing [12]. Since it is easier and cheaper to 
use a number instead of a relation as a hash table entry, we “identify” a relation 
with a number. The user specifies how many relations he expects to be in the 
input file(s) (maxrelsinp). This figure is used to choose the size of the in-memory 
tables needed during the pruning algorithm. The program reads in relation after 
relation. In order to detect duplicates, the program maps each relation (a, b) to 
an integer between 0 and 2®^ — 1. The mapping function, h = h{a,b), should 
be nearly injective since relations mapped to the same value will be treated 
as duplicates. It is rather easy to construct such a function, since even a huge 
amount of relations, say 200 million (for RSA-155 we had to handle 124.7 million 
relations), is small compared to the 2®"^ possible function values. With 64 bits 
for the function value we expect about 

f2 ■ 10® 

V 2 

2®4 

false duplicates, which means that there will hardly be any false duplicates. 
With 32 bits only, this number would amount to about 4.7 • 10®, which is a fair 
proportion of all relations. 

The function h{a, b) is defined as follows. It takes values of a and b up to 2®®. 
Put iT = [tt • lO^^J and E = [e ■ lO^^J . We have gcd(iT, E) = 1. Define 

H{a, b) = Ila -\- Eb. 

If El{ai, 5i) = i 7 (o 2 , 62 ) and (oi, 5i) yf ( 02 , 62 ) we have 

01—02 E 

61-62 n 

which is impossible, since |o| and | 6 | are known to be much smaller than 11/2 and 
E/2, and gcd(iT, E) = 1. Define h{a, 6 ) = H{a, 6 ) mod 2®®. Since E[ is injective, 
false duplicates for h can only come from the truncation modulo 2 ®®. 

The function values of h again are mapped by a hash function into a hash 
table. If the user has specified mergelevel 0, the non-duplicates are written to 
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the output file whereas, if the user has chosen mergelevel 1 , the non-duplicate 
relations are memorized in a table for further processing, while considering only 
the large prime ideals. In the sequel, we shall call this table the relation table. 



Singletons. If both polynomials / and g split completely into distinct linear 
factors modulo a prime p which does not divide the leading coefficients, we get 
a so-called free relation corresponding to the prime ideal factorization of the 
elements p = p — Oa and p = p — OP of norm IVq(q)/q(p) = F{0,p)/ci = p^^^ij) 
and IVq(/ 3 )/q(p) = G( 0 ,p)/c 2 = p'^^s(g)^ respectively. Approximately 1 /( 5 / • pg) 
of the primes offer a free relation, where pf and pg are the orders of the Galois 
groups of the polynomials / and g, respectively [10]. The free relation (p, 0) 
is added to the relation table only if all prime ideals of norm p appear in the 
relation table. 

Next, a frequency table is built for all occurring prime ideals which is adjusted 
as the relation table changes. The relation table is then scanned circularly and 
relations containing an ideal of frequency 1 (singletons) are removed from it. 
The program executes as many passes through the table as is needed to remove 
all singletons. 

At the end of the pruning algorithm we would like the remaining number of 
relations to be larger than the total number of prime ideals. Therefore we need 
to reserve a surplus of relations for the small prime ideals: Per polynomial, the 
number of prime ideals below filtmin is approximately 7r(f iltmin), i.e., the 
number of primes below filtmin, see [14]. Consequently, we require a surplus 
of approximately (2 — {pf ■ Pg)~^) ■ 7r(f iltmin) relations. If the required surplus 
is not reached we need to sieve more relations. 



Clique Algorithm. If there are sufficiently many more relations than ideals, 
the user may want to specify how many more relations than large ideals to retain 
after the pruning stage (keep). 

In [19, step 3] Pomerance and Smith eject excess relations by simply delet- 
ing the heaviest relations. However, as an alternative, they suggest to delete 
relations which contain many primes of frequency 2. Our approach is similar to 
this alternative. The algorithm we use is called clique algorithm, since it deletes 
relations that stick together. 

Consider the graph with the relations from the relation table as nodes. We 
connect two nodes if the corresponding relations would be merged in a 2 -way 
merge. The components of the graph are called cliques. The relations in a clique 
are close to each other in the sense that if one of them is removed, the others 
will become singletons after some steps and are therefore useless. 

The clique algorithm determines all the cliques, evaluates them with the 
help of a metric and at each step keeps up to a prescribed number of them in 
a priority heap [12, page 144], ordered by the size of a metric value. The metric 
being used weighs the contribution from the small prime ideals by adding 1 for 
each relation in the clique and 0.5 for each free relation. The large prime ideals 
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which occur more than twice in the relation table contribute 0.5^“^ where / is 
the prime ideal’s frequency. This way we “penalize” ideals with low frequency. 
Relation-sets containing many ideals with low frequencies are more likely to be 
deleted than those containing mainly high frequency ideals. By deleting these 
low-frequency relation-sets we hope to reduce especially low frequencies even 
more and get new merge candidates. 

Finally, the relations belonging to cliques in the heap are deleted from the 
relation table. When deleting relations we decrease the ideal frequencies of the 
primes involved. Singletons may arise and we therefore continue with the sin- 
gleton processing step. The clique algorithm may be repeated if the number of 
excess relations does not approximate keep sufficiently. 

After duplication, singleton and possibly clique processing the relations are read 
again and only the non- free relations** appearing in the relation table are written 
to the output file. If the input files have grown in the meantime, the new relations 
are discarded. 



2.2 Merging 

First, we have a closer look at how merging works, which parameters can be 
given and at how to minimize the weight increase during a fc-way merge. Next, 
we give details about the implementation of the “controlled” merges. Finally we 
study the influence of merging on Block Lanczos’s running time. 

Merging aims at reducing the matrix size by combining relations. Through- 
out this section we give figures about weight changes in the matrix. These figures 
do not take account of possible other primes that may have been balanced inci- 
dentally during the same merge. 



Parameters mergelevel, maxpass, maxrels and maxdiscard. With the pa- 
rameter mergelevel the user specifies the highest k for which fc-way merges 
are allowed to be executed. The user fixes the maximum number (maxpass) of 
shrinkage passes to execute. During a shrinkage pass, all large primes are checked 
once and possibly merged, see [8, section 7] for more details. 

The simplest case is the so-called 2- way merge. A prime ideal / is unbalanced 
in exactly two relations, ri and V 2 , and we combine the relations into the relation- 
set ri -|- T 2 . As a result, we have one fewer column (ri and V 2 disappear, ri -|- V 2 
enters) as well as one fewer row (prime ideal I) and the total weight has thereby 
decreased by 2. 

In general, if a prime ideal / is unbalanced in exactly k relations (fc > 2)* * *, 
we can choose k — 1 independent relation pairs out of the possible ^ 2 ) pairs. For 
example, if fc = 3, there are 3 possible ways to combine the 3 relations involved, 
ri, T 2 and r^, to a couple, namely ri -|- r 2 , T 2 + and ri -|- r^. Each one can be 



Free relations will be generated during the merge stage again. 
The case k — 1 denotes a singleton which would be deleted. 
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obtained from the other two, for instance ri + = (ri + r 2 ) + {r^ + r^) as all 

the prime ideals of are balanced since V 2 appears twice. 

After the merge, the prime ideal / is balanced. Its corresponding row has 
disappeared from the matrix. The total gain of every merge consists in fact in 
one fewer column and one fewer row. The drawback of merging is, of course, 
matrix fill-in. A 2- way merge causes no fill-in at all, we even have 2 entries fewer 
in the matrix. However, a fc-way merge, fc > 3, causes the matrix to be heavier by 
about the weight of fc — 2 relations minus the 2(fc — 1) entries that disappeared. 

If the matrix is going to be “lopsided” , i.e., if it has many more relations than 
ideals, it is useful to drop heavy relation-sets. The program therefore discards the 
ones which contain more relations than the user-determined bound maxrels.l 
The user may specify maxdiscard, that is, the maximum number of relation- 
sets to be dropped during one filter run. Once maxdiscard has been reached, 
fc-way merges, fc > 3, are inhibited. 



Minimizing the Weight Increase of a fc-Way Merge. Which fc — 1 of the 
possible ^ 2 ^ relation pairs should be chosen in order to achieve the lowest weight 
increase? First of all, each relation has to appear in at least one relation couple, 
that is, we need to form independent relation sets, in order not to loose data. 
Secondly, we focus on minimizing the weight increase. In the beginning, when all 
relations are true single relations, we usually achieve the lowest weight increase 
by choosing the lightest relation (pivot) and combining it with the remaining 
fc — 1 relations. We call this pivoting. More precisely, this happens always when 
no additional prime ideals except for the prime ideal / become balanced in any 
of the candidate relation couples. If we assume the pivot relation to be r^, the 
weight increase Aw will be exactly 

Aw = (fc — 2)w{vk) — 2(fc — 1). (1) 

The choice becomes more complicated, when additional prime ideals get bal- 
anced, especially when we are merging already combined relation-sets. For ex- 
ample, consider the following 5 relations, which are candidates for two 3-way 
merges with the prime ideals / and J: 

ri : I and f — 1 other prime ideals 
T 2 : / and f — 1 other prime ideals 
V 3 : I, J and v — 2 other prime ideals 
T 4 : J and f — 1 other prime ideals 
rs : J and f — 1 other prime ideals 

For the sake of simplicity, we assume that all the relations have the same weight 
V and do not share other primes except for / and J . Imagine, is used as a 



^ We weigh a free relation less than 1 (we used 0.5), because, even if it may have 
several large primes, it should have less total weight. 
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pivot relation to eliminate I. We get 

ri + T 3 : J and 2w — 3 other prime ideals 

^2 + '^ 3 , ■ J and 2v — 2> other prime ideals 

T 4 : J and w — 1 other prime ideals 

rs : J and f — 1 other prime ideals 

Now J appears 4 times, so we need a 4- way merge to balance it. For the elimina- 
tion of J the two relations r 4 and seem the best pivot candidates in a 4-way 

merge, since they have lowest weight. However, pivoting with results into 

(ri -I- ra) + : 3v — 4 prime ideals 

{i "2 + ra) + : 3v — 4 prime ideals 

f 4 + : 2v — 2 prime ideals 

with total weight 8v — 10, whereas 

(ri -I- ra) -I- (r 2 -I- ra) : 2v — 2 prime ideals 
(ri -I- ra) -I- rs : 3u — 4 prime ideals 

^4 + : 2v — 2 prime ideals 

ends with weight 7u — 8^. When v > 2 we have 8u— 10>7u — 8 which indicates 
that we should not stick to pivoting for all the merges. 

The problem of minimizing the weight increase can be stated using graphs. 
The vertices are given by the k relations which are candidates for a fc-way merge 
and the ^ 2 ) edges between them represent possible merges. The edge between 

two nodes Vj and rj has weight w{ri + rj). Given this weighted graph we wish 
to select a tree with minimum total weight. The solution is called a minimum 
spanning tree [11, page 460]. This problem is a well-known problem of combina- 
torial optimization. In order to solve it we use the algorithm as formulated by 
Jarnik [9, pages 46-47]. 



Implementation of “Controlled” Merges. We limit the weight increase of 
a single merge by requiring that a merge should not add more than a prescribed 
number, rrimax, of original relations to the matrix. We give all the initial re- 
lations the same weight (except for free relations that weigh one half), which 
is reasonable since the relations are the factorizations of numbers of about the 
same size. 

Let us consider k relation-sets which are candidates for a fc-way merge. The 
individual relation-sets may contain several original relations. Suppose the light- 
est candidate relation-set has j relations, where free relations count for 0.5. Let 
c be the number of relation-sets with exactly this minimum number j of rela- 
tions. Shrinkage pass 1 starts with m = 1 and we subsequently augment m up 
until rrimax and allow for the fc-way merge when (k — 2)j < m — (c— l)/2. The 

* The latter situation is also achieved when first using ri as a pivot and then doing a 
3- way merge with pivot relation rs. 
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m gives the maximum weight increase (in number of relations) allowed during 
a merge. We introduced c in order to postpone some merges and do the ones 
where the best way to merge is clear cut first. Since we are still interested in 
doing lower weight merges before higher weight merges we increase m only every 
other shrinkage pass and set c = 1 during these shrinkage passes. In most of the 
runs we had rrimax = 7, but we tried rrimax = 8 as well. Solving the inequality 
{k — 2)j < rrimax for k gives k < +2. It follows that, with rrimax = 7, merges 

with ordinary relations (j = 1) are limited to prime ideal frequency 9 whereas 
free relations (j = 0.5) can be used in merges up to prime ideal frequency 16. For 
the factorization of RSA-155 we performed merges up to prime ideal frequency 8. 

Table 1 shows the maximum number of relations a pivot relation-set may 
consist of, for rrimax = 7 and 8. Even if we are not pivoting, we ask at least one 
relation not to contain more relations than this bound. 
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Table 1. Allowed number of relations in pivot relation-set for fc-way merge 



Influence of Merging on Block Lanczos’s Running Time. Given anmxn 
matrix, n > m, of total weight w, the running time estimate of Block Lanczos 
is given by 0{wn) + 0{n'^) [16]. Both terms grow with n, so we will focus on 
reducing n. If we manage to reduce n by a certain factor while w does not grow 
by more than this factor, we will get a running time reduction, independently of 
the constants in the two terms. Moreover, we predict the constant in the O(n^) 
term to be the larger one. Therefore, it is natural to write the running time as 

0{{w + Cn)n) (2) 

with C > 1. Since we do not need absolute running times, we drop the O-sign 
and use the function t(n, w) = (w + Cn)n. The larger the constant (7, the more 
it will be convenient to reduce the matrix size. The constant depends on the 
implementation, for example on the number of bits per vector element {K) used^. 



Montgomery [16] gives the formula OiwnjK') + 0{n^) for the running time. 
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Montgomery (personal communication) at first estimated the constant C to be 
about 50. For some approximate values of C see Table 7 or Table 2. 

Let us determine a bound for the weight increase Aw such that a merge 
causing an increase below this bound still is beneficial to the running time. The 
condition for Aw becomes 



t{n — 1, w + Aw) — t{n, w) < 0. (3) 

Inequality (3) is equivalent to 

0 > n ((1 — 2n)C — w + {n— l)Z\w) = (n — l)(—2Cn — w + nAw) — w — Cn. 



The inequality is satisfied if Aw < 2C + ^. It follows that the allowed weight 
increase grows with C and the average column weight ^ . That means that denser 
matrices allow heavier merges than sparser matrices do. 

Let us calculate a limit for the pivot relation weight j of a general fc-way 
merge, fc > 3. According to equation (1) we require 

Aw={k- 2)j - 2{k - 1) < 2C + ^. 



which results into 

2C+H + 2(fc-l) 

7 < ^ ^ 

k-2 



(4) 



In Table 2 we report the allowed pivot relation weights for merges up to prime 
ideal frequency 10. We chose ^ = 30 (typical after applying only 2- and 3- way 
merges) and — = 50 (typical — of many of our final matrices). The horizontal 
lines divide between above and below 
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Table 2. Allowed pivot relation weights for fc-way merge 



From Table 2 we can see that 3-way merges can be done with rather heavy 
pivot relations; even for (7=1 and ^ = 50 the allowed weight exceeds ^ . Denser 
matrices allow also for denser pivot relations. 
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By substituting ^ for j in (4) we can derive a condition for when to do fc-way 
merges for fc > 3 with an average weighing pivot relation: 

w 2C + 2{k-l) 

n< k-3 ® 

The analysis for fc = 3 has to be done separately, we require (3) for Aw = ^ — 4 
By reorganizing the terms we get —4 (n — 1) — ^ — C {2n — 1) < 0 which is 
always satisfied. This means that 3-way merges with an average weight pivot 
relation are always profitable, independently from the density of the matrix or 
the constant C . 

Table 3 gives the allowed average weights when merging with an average 
weight pivot relation. If we assume (7 < 50 and we apply the merges in ascend- 
ing order of prime ideal frequency, 6-way merges with average weighing pivot 
relations will not be worthwhile because after the 5-way merges we have seen in 
practice ^ to be around 50, which is higher than the maximum value of 35. 




Table 3. Allowed average weights for fc-way merge 



3 Other Methods in the Literature 

We would like to mention two articles about similar filter strategies. These are 
“Solving Large Sparse Linear Systems Over Finite Fields” of LaMacchia and 
Odlyzko from 1990[13] and “Reduction of Huge, Sparse Matrices over Finite 
Fields Via Created Catastrophes” of Pomerance and Smith from 1992[19]. Their 
strategies are similar to each other but differ in some points. Both were de- 
signed to reduce the initial data to a substantially smaller matrix. This matrix 
was allowed to be fairly dense since it was going to be processed by Gaussian 
elimination afterwards. In contrast, the purpose of our method is to reduce the 
matrix size but still keep it sparse in order to take advantage of the Block Lanczos 
method. They were dealing with matrices of size up to 300K, we with matrices 
of size up to 7M. Each reflects the maximum size that could be handled at the 
time. 
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Both other methods executed their operations on the matrix itself whereas 
we dealt with the raw relations. We identified relations with columns in the 
final matrix whereas they identified relations with rows. Nevertheless, for an 
easier comparison, we will stick to identify relations with columns in the present 
description. 

They operate only on part of the matrix (active rows) where no fill-in takes 
place. The operations must be memorized in order to be repeated on the complete 
matrix afterwards. LaMacchia and Odlyzko store the history in core, whereas 
Pomerance and Smith keep a history file. 

We will distinguish between the pruning and merging step, as in the descrip- 
tion of our method. The weight they look at is only the weight of the active 
primes at that moment. 

The pruning step does differ from our approach only in how to delete ex- 
cess relations. Duplicates and singletons are removed as soon as possible, as in 
our approach. Pomerance and Smith choose to remove the excess immediately, 
whereas LaMacchia and Odlyzko remove the excess just before the “collapse” or 
“catastrophe” during the merge step. Both decide to drop the heaviest relations, 
but Pomerance and Smith indicate that one might try other strategies (as we 
did). 

In the beginning of the merge stage, a small number of rows (the heaviest, 
which correspond to small primes) are declared inactive. Merges are done by 
pivoting with columns that have only one 1 in the active part. There is no fixed 
limit for the prime ideal frequency up to which to merge. Once all possible merges 
have been done and there are still I’s in the active part, more rows (again the 
heaviest) are declared inactive and the merge step is repeated. This is repeated 
until the active part collapses. This procedure leads to very heavy matrices. To 
overcome this, LaMacchia and Odlyzko for example, extend the inactive part 
considerably after it has reached a certain critical size. This way fewer merges 
can be executed and the fill-in is confined. Nevertheless, the matrices still have 
high column weights: the lightest example given by LaMacchia and Odlyzko 
has an average of 115 entries per column for a 6.0 • 10^ columns matrix which is 
much denser than our densest matrix, the 6.3 • 10® columns matrix from Table 11 
having an average 81 entries per column^. 

Initially, for a sparse matrix, merges are done with very light columns, since 
the inactive part is small and cannot contain many I’s. Further on, pivot rela- 
tions can be very heavy: very probably, the single 1 in the increasingly smaller 
active part mostly represents a large prime and goes together with many small 
prime factors, since all polynomial values are about the same size (Pomerance 
and Smith try to overcome this by also allowing merges with pivot columns hav- 
ing two I’s in the active part of the matrix.). Moreover, they do not make a 
distinction between “original” pivot relations and already merged ones, which 
can be substantially heavier. 



^ The column weight 70 given in Table 11 corresponds to the matrix obtained when 
dropping the prime ideals of norm below 40. 
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In our merge procedure we also merge with already merged relations, but 
this happens in a controlled way. We limit the number of original relations which 
can be added during a single merge. We also minimize the fill-in per merge by 
using a minimum spanning tree algorithm instead of the simpler pivoting, see 
Section 2.2. But here we also have to say, that we cannot guarantee to always 
get the cheapest merge, because we count the contribution from the large prime 
ideals but only estimate the contribution from the small prime ideals. 

In 1995, Thomas Denny proposed a Structured Gaussian elimination prelim- 
inary step for Block Lanczos [7] . He estimated (7 = 1 for his own Block Lanczos 
program. We therefore also included (7 = 1 in Tables 2 and 3. 



4 Experimental Results 

The experiments were done with two versions of our program filter. Both of 
them include pruning facilities. 

The first version was capable of doing merges up to prime ideal frequency 5 
and corresponded to the old program [8, section 7] if invoked with mergelevel 2 
or 3. With the first version the user needed to specify when to start with the 4- 
and 5-way merges. For example, in the tables about filter runs (Tables 5, 8 and 
10) the notation 4(x) in column mergelevel means that 4-way merges started 
X shrinkage passes after 3-way merges started. 5(x-y) means that 4-way merges 
started x shrinkage passes after 3-way merges did, and 5-way merges started y 
shrinkage passes later than 3- way merges. 

The present filter version does not need this information any more. It can 
do merges up to prime ideal frequency 18. The merges are done in order of weight 
increase (measured in numbers of original relations). All runs except RSA-155’s 
B6 had rrimax = 7. 

Table 4 gives an overview of all pruning activities in our experiments for 
RSA-140, R211 and RSA-155. All the figures are in units of a million. With 
prime ideals we mean prime ideals above lOM; we need to reserve an excess 
of 1.3M relations for the small prime ideals. The non-duplicate relation counts 
differ so much due to the use of different large prime bounds. Apparent errors 
are due to rounding values to units of one million. 

The figures in Tables 5-11 are given in units of a million (M) or a thousand 
(K). We labeled the experiments with capital letters. All experiments with the 
same letter started with the same mergelevel 1 run. 

In Tables 5, 8 and 10, columns 2-6 are input parameters. Column 7-10 are 
results: column “sets” gives the number of relation-sets remaining after the run, 
column “discarded” gives the total number of relation-sets which were discarded 
during the run. “excess” gives how many more relations than the approximate 
total number of ideals we retained. It indicates how many more relations we 
might still throw away in a further run. “not merged” gives the number of large 
prime ideals of frequency smaller or equal to mergelevel among the output 
relations. For the runs with the new version we also report the number of output 
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number being factored 




RSA-140 


R211 


RSA-155 




experiment 




A B 


A B 


ABC 


D 


raw relations 


(1) 


65.7 68.5 


57.6 


130.8 




duplicates 


(2) 


10.6 11.9 


10.6 


45.3 




non-duplicates 


(3) = (l)-(2) 


55.1 56.6 


47.0 


85.5 




free relations 


(4) 


0.1 0.1 


0.8 


0.2 




prime ideals 


(5) 


54.2 54.7 


49.5 


78.8 




excess 


{6) = (3) + (4)-{5) 


1.1 2.0 


-1.7 


6.9 




singletons 


( 7 ) 


28.5 28.2 


26.5 


32.5 




relations left 


(8) = (3) + (4)-(7) 


26.8 28.5 


21.3 


53.2 




prime ideals left 


(9) 


21.5 22.6 


18.5 


42.6 




excess 


(10) = (8)-(9) 


5.2 6.0 


2.8 


10.6 




clique relations 


(11) 


17.6 18.7 


7.4 0 


34.1 33.0 29.6 22.9 


relations left 


(12) = (8)-(11) 


9.2 9.8 


13.9 21.3 


19.1 20.2 23.6 30.3 


prime ideals left 


(13) 


7.8 8.1 


12.2 18.5 


17.4 18.2 20.6 25.3 


excess (=keep) 


(14) = (12)-(13) 


1.4 1.7 


1.7 2.8 


1.7 2.0 3.0 


5.0 



Table 4. summary of mergelevel 0 and 1 runs 



relation-sets made of one single relation since among those could be candidates 
for future high-way merges. 

The Block Lanczos code typically finds almost K dependencies [16], where 
K is the number of bits per vector element. This enables us to drop the heav- 
iest rows which leads to substantially lighter matrices^ . We dropped the rows 
corresponding to prime ideals of norm smaller than 50 for R211, whereas for 
RSA-140 and RSA-155, which have both exceptionally many small prime ideals, 
we omitted the prime ideals of norm smaller than 40**. In addition, the Block 
Lanczos code truncates every m x n matrix by default to m x (m + K + 100). 

The tables featuring matrix data (Tables 6, 9 and 11) are made of two parts. 
In the first part we state the real size (m x n), weight (w) and average column 
weight (^) of the matrices built. The numbers between two lines express the 
changes in size (number of columns) and weight from one matrix to the smaller 
one as percentages. Note that a i% decrease in matrix size makes the term 
wn shrink as long as the weight does not increase by more than which 

is slightly larger than i%. The second part shows the effective weight {weff) 
after truncating the matrix to size m x (m + K + 100), the effective average 
column weight ( and the Block Lanczos timings from a Cray 090 and 
a Silicon Graphics Origin 2000. The timings can vary substantially according to 

I In particular, all quadratic character rows are omitted. The pseudo-dependencies 
being found for this reduced matrix must be combined to real dependencies after- 
wards. 

** These figures match with the implementation for K — 64. For K = 128, we could 
even have dropped the prime ideals up to norm 180. The resulting lighter matrices 
would have led to shorter timings for that implementation. However, for simplicity, 
we used the same matrices for both the K — 64 and the K — 128 versions. 
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the load on the machines (other jobs interacting with ours): time differences of 
20% are not unusual. Aiming at a fair comparison we tried to run the matrices 
at times with comparable load. In our tables, comparable timings are written 
in the same column. Only one Block Lanczos job per number was completely 
executed. All times in the tables are extrapolations: we did a short run, took 
the time of the fastest iteration and multiplied it by the number of iterations 
(m+ A:+ 100)/(A:-0.76), see [16]. 

RSA-140 

This 140-digit number was factored on February 2, 1999. The experiment se- 
ries A started with 65. 7M raw relations, B with 68. 5M from 5 different sites. 
We removed 1.4M and 1.6M duplicates, respectively, with mergelevel 0 runs 
on each contributor’s data. The experiments in Table 5 start with the remaining 
64. 3M respectively 66. 9M relations having 54. 2M and 54. 7M large prime ideals, 
respectively. After the pruning step (with f iltmin= lOM) we need an excess of 
y| 27 t( 10 M) = 1.3M for the small prime ideals. For a summary of mergelevel 0 
and 1 runs, see Table 4. 

In this paragraph we only describe experiment series A. The mergelevel 1 
run on the whole bunch of data removed another 9.2M duplicates and added 
O.IM free relations for large primes. Note, that at this point the excess 64.3M — 
54. 2M — 9.2M -I- O.IM = I.IMA was less than the needed 1.3M. The excess 
was sufficient only after removing the singletons, when we were left with 26. 8M 
relations having 21. 5M large prime ideals. The clique algorithm removed a total 
of 17. 6M relations to approximate the excess of 1.4M = 9.2M — 7.8M. 

The factorization was done using matrix A 1.1 which took 10 Oh on the Cray. 
Only 2- and 3-way merges were performed, because the code for higher than 
3-way merges was not ready by then. For logistic reasons we had built the matrix 
before we received all the data. 

With the complete data (experiment series B) the excess was enough from 
the beginning. Furthermore, a matrix constructed from this data by applying 
the same filter strategy as for Al.l would have performed better than Al.l as 
one can imagine when comparing Al. 1.2.1 to B1.2: both did merges up to prime 
ideal frequency 5 and the latter is smaller in size and weight. 

We also tried mergelevel 8 (B2) with rrimax = 7 which was introduced 
only just before the factorization of RSA-155. The program stopped with fc-way 
merges, fc > 3 at shrinkage pass 10 after having deleted 381K relations. This 
means that only merges with a maximum weight increase of 6 original rela- 
tions had been done. Matrix B2 beats the mergelevel 5 matrix of the same 
series (B1.2). 

In Table 6 one can see from the percentages that each size reduction should 
have a favourable effect on Block Lanczos’s running time which is confirmed by 
the time column. 

The apparent arithmetical error is due to rounding all numbers to units of a million. 
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These experiments confirm our idea of the advantage of higher-way merges. 
They show that collecting more data than necessary is recommendable. It does 
not become clear, however, how much excess data one should keep after the 
pruning step. 



4^ 

a 

a 

a 

X 


1 — 1 
CD 
> 
CD 
1 — 1 
CD 
bO 

CD 

B 


•H 

B 

rH 

•H 

MH 


maxdiscard 

maxrels 

maxpass 


sets 


cd 

CJ 


excess 


not merged 


A 


1 


lOM 


keep 1.4M 


I9.2M 46 040K 


90K 


- 


A1 


2 


lOM 


- 4.0 6 


6.0M 


54K 


36K 


59 


Al.l 


3 


lOM unlim. 10.0 10 


4.7M 


3K 


33K 


0 


Al.1.1 


4(0) 


lOM 


20K 10.0 10 


4.2M 


20K 


13K 


243K 


Al.1.2 


4(0) 


lOM 


20K 12.0 10 


4.0M 


14K 


20K 


0 


Al.1.3 


4(0) 


lOM 


20K 11.0 10 


4.0M 


20K 


13K 


48K 


Al. 1.2.1 


5(0-0) 


8M 


17K 15.0 10 


3.5M 


17K 


4K 


0 


B 


1 


lOM 


keep 1.7M 


I9.8M 46 906K 384K 


- 


B1 


4(5) 


lOM 


300K 8.0 12 


4.3M 


170K 208K 


6K 


Bl.l 


5(1-3) lOM 


200K 11.5 10 


3.6M 


85K 128K 


IK 


B1.2 


5(1-3) lOM 


200K 10.5 10 


3.4M 


200K 


14K 


28K 


B2 


8 


lOM 


375K 8.0 15 


3.3M 


383K 


IK 909K/455K 



Table 5. RSA-140 filter runs 



exp. 


matrix size % 


weight % 


col.w. 


Weff 


col.w. 


Cray 


SGI “ 


Al.l 


4 671K X 4 704K 


151. IM 


32.1 


147.4M 


31.5 


75h 


59d 24d 


Al.1.1 


4180K X 4193K 


163.1M 


38.9 


161. 3M 


38.6 


65h 


56d 22d 


Al.1.3 


3 999K X 4012K 


168.7M 


42.0 


166. 8M 


41.7 


63h 


54d 21d 


Al.1.2 


3 960K X 3 980K 


171. IM Y 


43.0 


168. IM 


42.4 


62h 


53d 20d 


Al. 1.2.1 


3 504K X 3 507K 


191.3M 


54.5 


190. 8M 


54.4 


56h 


51d 18d 


B1.2 


3 380K X 3 394K 


178.8M 


52.7 


176. 8M 


52.3 


51h 


46d 16d 


B2 


3 285K X 3 286K 


182.1M 


55.4 


182. OM 


55.4 


50h 


43d 15d 



Table 6. RSA-140 matrices 



“ The second column gives timings from the K = 128 implementation. 



With each timing column, we fitted a surface t = sin^ + S 2 nw to the 
points (n, w, t). The fits were done by gnuplot’s implementation of the nonlin- 
ear least-squares (NLLS) Marquardt-Levenberg algorithm. The quotient si/s 2 
corresponds to the C from (2). Table 7 gives some possible values for C. 
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Block Lanczos implementation 


Si 


S2 


C 


vectorized Cray code with K — 64 


1.84 


±0.06 


0.0499 ±0.0014 


37 ±2 


SGI code with K = 64 


0.86 


±0.14 


0.060 ±0.003 


14 ±3 


improved SGI code with K = 128“ 


0.69 


±0.08 


0.0140 ±0.0018 


49 ±12 



Table 7. C values for different Block Lanczos implementations 



“ This version ‘under development’ by Montgomery is being optimized for cache usage 
rather than vectorization. It is being redesigned to allow parallelization, but we used 
only one processor. 



C = 14 is much smaller than we had initially expected. According to Table 2, 
with C = 14 and assuming ^ = 30 we have that 4-way merges are convenient 
with pivot relations up to weight 31, which is slightly above average whereas 
5- way merges should be done with lighter than average (max. 21 entries) pivot 
relations. When assuming ^ = 50 the maxima are higher but below average also 
for 4-way merges. 

Why then did the matrices, which were constructed by more or less brutally 
doing all possible 3-, 4- and 5-way merges^^ , perform better than we would expect 
from looking at the figures in Table 3 and 2? It seems most merges were able to 
find a pivot relation with much smaller weight than average. Furthermore, we 
must consider that the inequalities (4) and (5) do not take account of the weight 
and size reduction obtained by discarding relation-sets which are made of more 
than maxrels relations. Some benefit also comes from the minimum spanning 
tree algorithm. 

With C = 49 and ^ = 30, even above average 6- way merges can be beneficial. 

R211 

The following two tables give data concerning filter experiments with the special 
211-digit number R211:= (10^^^ — l)/9, which is a so-called “repunit”, since all 
its digits are 1. It was factored on April 8, 1999. Five sites produced a total of 
57. 6M raw relations. 1.2M duplicates were removed during mergelevel 0 runs 
on the individual data. The experiment series A and B both started with the 
remaining 56. 4M relations having 49. 5M prime ideals of norm above lOM. This 
means that we had 6.9M more relations than prime ideals which seemed to be 
enough since we needed to reserve y|7t( 10M) = 1.3M more relations accounting 
for the small prime ideals. Unfortunately, the mergelevel 1 run on the complete 
data set revealed 9.4M duplicates. The remaining 47. OM relations plus 0.8M free 
relations were less than the number of prime ideals. However, we did not need 
to sieve further since we had an excess after removing the 26. 5M singletons. The 
clique algorithm started hence with 21. 3M relations having 18. 6M prime ideals 
of norm larger than lOM, which is an excess of 2.8M. See Table 4. 

For Al. 1.2.1, all possible merges up to prime ideal frequency 5, for prime ideals of 
norm larger than 8M, had been performed. 




Strategies in Filtering in the Number Field Sieve 227 



Experiment series A gives the parameters and results of the filter runs that 
led to the matrix that was used to factor the number; it took 120 hours on the 
Cray. B shows a different approach, where we kept I.IM more relations than for 
A after the pruning step, leaving more choice for merging. 



4^ 

a 

s 

a 

X 

a; 


1 — 1 
CD 
> 
CD 
1 — 1 
CD 

bO 

U 

CD 

& 


l=! 

•H 

B 

rH 

•H 

MH 


maxdiscard 

maxrels 

maxpass 


t/2 




excess 


not merged 


A 


1 


lOM 


keep 1.7M 


I13.9M 33 839K 


433K 


- 


A1 


4(5) 


20M 


300K 6.0 10 


6.8M 


304K 


124K 


1637K 


Al.l 


5(5-10) 20M 


15K 12.0 15 


5.6M 


15K 


109K 


796K 


Al.l.l“ 


5(5-10) 


8M 


50K 15.0 15 


4.9M 


n.a. 


63K 


n.a. 


B 


1 


lOM 


keep 2.8M 


121. 3M 26 488K 


1484K 


- 


B1 


4(5) 


20M 1300K 6.0 10 


6.7M 


1310K 


206K 


1410K 


Bl.l 


5(5-10) 20M 


170K 12.0 15 


4.8M 


170K 


IIK 


97K 


Bl.1.1 


5(1-3) 


8M 


lOK 18.0 10 


4.6M 


4K 


8K 


2 


B2 


8 


10M 1400K 9.0 15 


4.7M 


1421K 


30K 1244K/925K 


B3 


8 


lOM 1400K 10.0 15 


4.5M 


1423K 


64K 


918K/777K 



Table 8. R211 filter runs 



“ This run was done with the flag regroup, which splits up existing relation-sets and 
does merges from scratch, which leads to different relation-sets. 



Both mergelevel 4 runs can actually be considered mergelevel 3 runs, since 
the maximum number of discards, maxdiscard, was reached before 4- way merges 
would have started. 



exp. 


matrix size % 


weight % 


col.w. 


Weff 


col.w. 


Cray 


SGI 


Al.l.l 


4 820K X 4 896K 


234.2M 


47.8 


221. 2M 


45.88 


118h - 97h 93h 


96d 


Bl.l 


4 863K X 4 877K 


223. 3M ^ 


45.8 


221. 3M 


45.92 


119h - 97h 95h 


97d 


B2 


4 723K X 4 754K ^ 


231.9M +1* 
231. 2M ^ 


48.8 


228. 2M 


49.10 


- 95h 93h 92h 


95d 


Bl.1.1 


4 661K X 4 670K 


49.5 


229. 3M 


49.60 


115h - 93h 91h 


95d 


B3 


4 503K X 4 569K 


247.5M 


54.2 


239. OM 


53.06 


- 90h - - 





Table 9. R211 matrices 



Experiment series B achieved smaller matrices than A. The reason must be 
the different keep values during the pruning stage. Experiment series A kicked 
out 7.4M relations with the clique algorithm whereas B kept all the excess re- 
lations, performed more merges and discarded more relations during the merge 
steps. We can conclude that for this data the best thing was to skip the clique 
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algorithm. This is strongly connected to the fact that we barely had enough 
relations. Sieving any longer would surely have led to smaller matrices. 

Matrix A 1.1.1 performed better than matrix Bl.l, which may seem counter- 
intuitive since Bl.l produced the smaller and lighter matrix. However, matrix 
Al.1.1 contained fewer rows (fewer prime ideals) than matrix Bl.l and due to 
the default truncation taking place in the Block Lanczos algorithm the effective 
Al.1.1 matrix was smaller in size and weight than the effective Bl.l matrix. 

At B2 we also tried mergelevel 8 while having rrimax = 7. maxdiscard was 
reached already at shrinkage pass 9 (with 15 possible passes) when the allowed 
weight increase was 5 original relations. The final matrix was larger than B 1.1.1. 
We had chosen maxrels too low. It was 9, compared to 18 in Bl.1.1. With 
maxrels 10 we achieved the desired reduction (B3). 



RSA-155 

The 155-digit number RSA-155 (512 bits!) was factored on August 22, 1999. A 
total of 130. 8M relations were collected from 12 different sites. 6.1M relations 
were removed in individual mergelevel 0 runs. Another 39. 2M duplicates where 
removed in a mergelevel 0 run on the whole amount of data. All the experiments 
below started with the remaining 85. 5M relations and its 0.2M free relations. 
Therefore, in contrast to the previous examples, the figures in the discarded 
column do not contain any duplicates. See Table 4 for details. 

Matrix B2 was used for the factorization. It took 225 hours on the Cray. 



experiment 


1 — 1 
CD 
> 
CD 
1 — 1 
CD 

bO 

CD 

B 


l=! 

•H 

B 

rH 

•H 

MH 


maxdiscard 

maxrels 

maxpass 


t/2 

0) 


o3 

CJ 


excess 


not merged 


A 


1 


lOM 


keep 1.7M 


I19.1M 66 593K 


385K 


- 


A1 


5(1-3) lOM 


370K 11.0 12 


7.1M 


370K 


15K 


67K 


B 


1 


lOM 


keep 2.0M 


120. 2M 65 531K 


684K 


- 


B1 


8 


lOM 


600K 9.0 15 


6.9M 


603K 


81K 1611K/764K 


B2 


8 


7M 


670K 9.0 15 


6.7M 


672K 


13K 1576K/716K 


B3 


8 


7M 


670K 10.0 15 


7.1M 


366K 


317K 1432K/744K 


B4 


16 


7M 


670K 9.0 15 


6.6M 


690K 


-5K 4130K/694K 


B5 


16 


7M 


670K 10.0 15 


6.8M 


482K 


193K 3 797K/562K 




18 


7M 


670K 10.0 15 


6.3M 


672K 


n.a. 


n.a. 


c 


1 




keep 3.0M 


123. 6M 62 092K 1682K 


- 


Cl 


8 


10M 1670K 8.0 15 


6.8M 


1675K 


7K 1710K/698K 


D 


1 


lOM 




I30.3K 55 402K 3 677K 


- 


D1 


8 




7.1M 


3 698K 


-20K 2118K/780K 



Table 10. RSA-155 filter runs 
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The experiments indicate that retaining more data (keep > 3.0M) after the 
pruning stage did not help to reduce the size of the matrix. 

Experiments B4 and D1 discarded too many relation-sets which is recogniz- 
able from the negative excess. 

In B2 merging was stopped at shrinkage pass 11, while m = 6. Since there 
were still many unmerged ideals in B2, we tried to make the matrix smaller 
by increasing maxrels in B3 which allows also relation-sets with 10 relations, 
which were deleted in test B2. But even after this run many potential merge 
candidates remained unmerged, although maxdiscard was not reached. This 
indicates that the weight increase of the merges was considered too high and the 
merges were subsequently not executed. Next, we tried mergelevel 16, which is 
the maximum prime ideal frequency you can have a merge with for rrimax = 7. 
Some reduction was achieved (B4 and B5). Finally, we took rrimax = 8 together 
with mergelevel 18 and maxrels 10. maxdiscard was reached during shrinkage 
pass 14, when m = rrimax ■ 



exp. 


matrix size % 


weight % 


col.w. 


Weff 


col.w. 


Cray 


B2 


6 699K X 6 711K 


417.1M 


62.2 


415. 5M 


62.0 


218h 


B6 


6 342K X 6 354K 


445. 3M 


70.1 


443. 4M 


69.9 


213h 



Table 11. RSA-155 matrices 



Matrix B6 is 5% smaller than B2 but also 7% heavier. With C = 14 we can 
expect to save 1 — 14^6 699 ^+ 6 !e 99 - 4 i 7 i ~ running time, which is too small 
a gain to accept the weight increase, whereas with C = 37 or C = 49 we may 
save 3% or 4%, respectively. The effective runs on the Cray {C = 37) indicate a 
saving of 2%. 

5 Conclusions 

We extended our previous filter program to allow higher- way merges and 
proved theoretically and practically that we can reduce Block Lanczos running 
time by performing higher- way merges. We determined limits for the weight of 
pivot columns. 

During a merge, instead of merging by pivoting we calculate a minimum 
spanning tree in order to assure minimum weight increase. 

A denser matrix allows for more weight increase during a merge than a lighter 
one: this means we can merge with denser pivot columns. Therefore we do the 
light merges before the heavier ones. 

We determined the ratio between the two terms characterizing the running 
time of Block Lanczos for different implementations. To which extent we can 
profit from higher- way merges depends on this ratio. We saw values ranging 
from 14 to 49. With the help of this constants we can estimate the running time 
of a matrix, given the running time of another matrix. 




230 



Stefania Cavallar 



Collecting more data than necessary is advisable. The clique algorithm en- 
ables us to get rid of excess data quickly and in a sensible way. It is a useful tool 
when having abundant excess. 
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Abstract. In this paper we develop new algorithms for factoring poly- 
nomials over finite fields by exploring an interesting connection between 
the algebraic factoring problem and the combinatorial problem of stable 
coloring of tournaments. We present an algorithm which can be viewed as 
a recursive refinement scheme through which most cases of polynomials 
are completely factored in deterministic polynomial time within the first 
level of refinement, most of the remaining cases are factored completely 
before the end of the second level refinement, and so on. The algorithm 
has average polynomial time complexity and logp)*^*-^^ worst case 

complexity. Under a purely combinatorial conjecture concerning tourna- 
ments, the algorithm has worst case complexity (71*°®*°®" log Our 
approach is also useful in reducing the amount of randomness needed to 
factor a polynomial completely in expected polynomial time. We present 
a random polynomial time algorithm for factoring polynomials over fi- 
nite fields which requires only logp random bits. All these results assume 
the Extended Riemann Hypothesis. 



1 Introduction 

It is well-known that polynomials over finite fields can be factored in random 
polynomial time [5,11]. However all attempts for finding a deterministic poly- 
nomial time algorithm for the problem have yielded only partial results so far. 
These results can be classified into two categories: those assuming the Gener- 
alized Riemann Hypothesis (GRH) or the Extended Riemann Hypothesis (ERH) 
and those that do not rely on any unproven assumptions. In general the uncon- 
ditional results are more restrictive than those assuming the GRH or the ERH, 
yet they are more difficult to come by. Assuming the GRH, the best result so 
far is a deterministic algorithm due to Evdokimov [7] which factors polynomials 
of degree n over a finite field Eg in logq)*^^^^ time. This result improves 

on the result of Ronyai[13] which solves the factoring problem in polynomial 
time when the degree of the input polynomial is bounded by a constant. We 
refer the readers to [4] for an extensive survey of other results and earlier works 
concerning deterministic factorization of polynomials over finite fields and re- 
lated problems such as deterministic construction of finite fields and finite field 
isomorphism problems. 
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In this paper we develop new algorithms for factoring polynomials over finite 
fields by exploring an interesting connection between the algebraic factoring 
problem and the combinatorial problem of stable coloring of tournaments. In 
this approach we associate a polynomial to be factored with a tournament on 
its roots. We design algebraic procedures that explore symmetry in the asso- 
ciated tournament and cause the polynomial to split into factors according to 
the symmetry classes. Further splitting, if necessary, is effected as deeper lev- 
els of symmetry is explored through algebraic means. The resulting algorithm 
can be viewed as a recursive refinement scheme through which most cases of 
polynomials are split completely at the first level within polynomial time, most 
of the remaining cases are split completely before the end of the second level 
refinement, and so on. 

The first level of our refinement scheme is a procedure which uses Ronyai’s 
method [13] as building blocks. The basic observation is that Ronyai’s method, 
when applied to a polynomial, groups the roots of the polynomial into factors 
according to scores in the associated tournament. By refining the method we 
obtain a procedure which implicitly performs stable coloring on the tournament. 
As combinatorial theory shows that most graphs decompose into singletons un- 
der a stable coloring, we can similarly show that most polynomials decompose 
into linear factors under this procedure. Should a non-linear factor survive the 
first level of refinement, it will be passed to higher levels of refinement where 
algebraic procedures are employed to explore higher levels of stable coloring on 
the tournament. 

The resulting deterministic algorithm has average polynomial time complex- 
ity and logp) worst case complexity, on input polynomials of degree n 

over Fp. Moreover, all but at most 2“”/® fraction of cases exit at the first level 
of the refinement scheme being completely factored. Then most of the remaining 
cases are split at the next level of refinement, and so on. The amount of time 
in going through the z-th level of refinement is bounded by P(rz*,logp) where 
P {., .) denotes a polynomial function. All cases are completely factored after no 
more than log rz/ 1.5 levels of refinement. 

The result assume the existence of quadratic nonresidues modulo p with 
absolute value polynomially bounded in logp, which is true assuming the ERH. 
In bounding the fraction of cases that may need to go on to higher level of 
refinement we need the additional assumption that n < log p/2. 

The tournament approach is also useful in reducing the amount of random- 
ness needed to factor a polynomial completely in expected polynomial time. We 
show that there is a random polynomial time algorithm for factoring polynomi- 
als over finite fields which requires only logp random bits. This is a significant 
reduction from the O(nlogp) random bits required by Berlekamp’s method. 

We will concentrate on polynomials all of whose roots are distinct and in a 
prime field Fp. The result can be extended to the general case by a well-known 
polynomial time reduction from the general case to the case we consider. We 
refer to [4] for details about this reduction. We assume the ERH throughout the 
rest of this paper. 
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Under a purely combinatorial conjecture concerning tournaments, we can 
show that the maximum level of refinement in our algorithm is as low as log log n, 
which implies that our algorithm has worst case complexity logp)*^*-^^. 

There are strong evidences for the conjecture which we discuss in Section 4. 

We remark that the method in this paper can be used to prove that polynomi- 
als of degree n over Fp can be factored completely in time P{n^^P\logp), where 
P is a polynomial function and S{p) is the size of largest transitive subgraph 
in the multicolor cyclotomic tournament over Fp. In light of this, an interesting 
open problem is to derive a sharp upper bound for S{p). 



2 The First Level of Refinement 

Let / G Fp[x] be a polynomial with all roots distinct and in Fp. We construct 
tournaments on Fp as follows. First assume that p= 3 (mod 4). For a,b G Fp, 
a has an arc to 5 iff a — 5 is a quadratic non-residue. This tournament is called 
quadratic residue tournament or Paley tournament [12]. Let 

/ = (a; - ai){x - 02 ) ■ ■ ■ {x - a„). 

We associate with / the subtournament induced by oi, ..., a„ in Paley tourna- 
ment. The score of a root at is the number of roots dominated by a^. Define the 
score polynomial of /, denoted S{f), as 

n 

( 1 ) 

i=l 



where bi is the score of a^. 

When p = 1{ mod 4), suppose p — 1 = 2^r' , where r' is odd. We construct 
a multicolor tournament over Fp called the cyclotomic tournament over Fp such 
that for i, j, s,t G Fp with i ^ j, s ^ t, the arc (z, j) has the same color as the 
arc (s, t) if and only if (z — j)’’ = (s — ty . We associate to / the sub-cyclotomic 
tournament on the roots of /. We define the score polynomial of / with respect 
to an arc color similar to above. (Actually we can replace 2^ by the smooth part 
of p — 1, and define generalized cyclotomic tournament.) 

An interesting observation is that the score polynomial S{f) can be computed 
in polynomial time using Ronyai’s method. We outline below how this can be 
done. For ease of presentation we assume p = 3 (mod 4). 

Let A be the companion matrix of /. Following Ronyai we construct the 
following linear space V spanned by Pi® Pj, i y j, where pi, - ■ ■ , pn are charac- 
teristic vectors of A, that is 

Api = aiPi. 

The linear transformation 



H = I ® A- A® I 
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acts upon V with Oj — aj as eigenvalues for i ^ j. The characteristic polynomial 
of C = H'" where r' = {p — l)/2 has —1 as a root. The invariant subspace 
of C + 1 contains all basic tensors of the form pi® pj, i yf j where Oj — aj 
is a quadratic nonresidue; that is, Oj dominates Qj. Hence as we construct this 
invariant subspace and construct the characteristic polynomial of the action of 
A® I on the subspace we get S{f). 

Theorem 1. There is an algorithm that given a polynomial / in Fp[x] of degree 
n, computes the score polynomial S{f) in polynomial time. 

This is a special case of Theorem 4, which we will introduce and prove later. 

From / and S{f) we can split / into factors each having roots of the same 
score with the following procedure. 

Algorithm 1. Input f with distinct roots in Fp. 

1. Calculate S{f) with respect to one of arc color (using Ronyai’s algorithm). 

2. Let h = S{f), /2 = /. 

3. While / 2 I /1 do fi = / 1 // 2 . 

4 . If f I = 1, quit the algorithm, otherwise output fi( gcdlyf\, f- 2 ). 

5. let /2 = gcd{fi,f 2 ). go to 3. 

We call a tournament regular if every vertex dominates the same number of 
vertices. For an irregular polynomial (tournament), after we apply the algorithm, 
we get several factors corresponding to the scores. However, we need not stop 
here. Suppose a factor is not regular (i.e. the roots of the factor do not induce a 
regular subtournament). Then it will be split when the algorithm is applied to 
it. Applying the algorithm to the product of two factors may also cause further 
splitting. These ideas lead to the following refinement procedure on a set of 
factors with disjoint sets of roots. 

Algorithm 2. Input a set of relatively prime polynomials {/i, fi, - ■ ■ , fn}, each 
with distinct roots in Fp. 

1. For 1 < i < n, apply the algorithm (1) on fi, let the set of output polynomials 
be Si. 

2. For 1 < z < j < n, apply algorithm (1) on fifj, for every output factor g, 
put gcd{g,fi) into Si, gcd{g,fj) into Sj. 

3. For every Si, if there are any two polynomial g,h G Vj, such that gcd{g, h) yf 
1, then remove g,h from Si and add gcd{g,h), g / gcd{g,h),h/ gcd{g,h) into 
S,. 

We apply Algorithm (1) to / and then apply Algorithm (2) to the set of fac- 
tors output by Algorithm (1). As we observe what is happening to the underlying 
tournament, we find that the process is very similar to the elementary refine- 
ment for undirected graphs [3]. The first procedure partitions the roots by score. 
Suppose Ci,C 2 ,‘ ■ ■ ,Ch form the partition. For all roots x, let Ni{x) denote the 
number of neighbors of x in Ci. In applying Algorithm (2) to the corresponding 
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set of factors, we first apply Algorithm (1) on Ci. This amounts to comparing 
Ni{x) for all x G Ci. Then we apply Algorithm (1) on CtCj. This amounts to 
comparing Ni{x) + Nj{x) for all x G CiU Cj. When we exit Algorithm (2), we 
have refined the partition in the following manner. Two roots x, y are now in 
the same class iff they are in same class before the refinement, and 

(A^i(a;), A‘ 2 (a;), • • • , Nh(x)) = (Ni{y), N 2 {y), • • • , Nh{y)). 

After we repeat Algorithm (2) at most n times, we will reach a point where 
the partition remains unchanged. At this point the partition of the roots is a 
stable coloring in the following sense. 

Definition 1. A partition of the vertex set of a tournament he into vertex class 

Cl, , Cm, is a level-one stable coloring if 

1. Ci, \ < i < m, induces a regular subtournament, 

2. For 1 < i, J < TO, for all u,v G Ci, u dominates the same number of vertices 
in Cj as v does. 

At this point we have completed the description of the first level of refinement 
in our algorithm for factoring polynomials over Fp. It is interesting to observe 
that after the level-one refinement, each factor is the union of some vertex orbits 
under the automorphism group of the tournament. 

Babai and Kucera proved in [3] that almost all graphs can be decomposed 
to singletons by only two refinement steps. We can prove a similar result for 
tournaments. 

Lemma 1. Let T he a random tournament on n vertices selected from the uni- 
form distribution over the set of labeled n-tournaments. The probability that T 
cannot he factored into singletons by the refinement is less than 2“”/®. 

See [6] for proof of the lemma. Based on the lemma and [8] we prove the 
following 

Theorem 2. The fraction of polynomials over Fp with degree n < logp/2 that 
cannot he split completely by the first level of refinement is less than 2“”/®. 

Proof. If a separable polynomial / has all roots on Fp, its root set will induce 
a sub-tournament in Paley tournament. On the other hand, every induced sub- 
tournament in Paley tournament corresponds to a completely splitting separable 
polynomial over Fp. It was proved in [8] that every labeled tournament (graph) of 
order n occurs roughly as frequently as it should as induced subtournament (sub- 
graph) in Paley tournament (graph), namely, with probability (1 o{\)) 

when n < (logp)/2. Hence the theorem follows from Lemma 1. 

It can be shown that if the first level refinement cannot split f{x) = 
with degree n < ^/logp completely, then the probability that it cannot split 
f([(a; — (ui kfi) completely is less than 2“°”, if k is uniformly picked up from 
Fp. From this we prove 
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Theorem 3. There is a randomized algorithm using only logp many random 
hits to split a polynomial of degree n over Fp completely in expected polynomial 
time when n < \/logp. 

Proof, (oi + kY — {cLj + kY = (oi — aj){at + Oj + 2k). Fix oi, a 2 , • • • , a„, w.l.o.g, 
assume for any i<j,u<v,Qi + aj Y + cLv except when i = u, j = v. Let k 
be a random variable, uniformly taking value from set 

{fcjfor any i Y j, cLi + cLj + 2k Y 0}- 

Let X : F* ^ {1, —1} be the character which sends x to x^~ , then x(ai + 02 + 
2k), x(oi + 03 + 2k), ■ ■ ■ , x(an-i + On + 2k) is a random 1,-1 sequence with size 
logp/2 with uniformly distribution [8]. Hence (oi + fc)^, (02 + fc)^, • • • , (a„ + fc)^ 
induce all subtournaments of the Paley tournament with uniformly distribution. 



3 The Second and Higher Levels of Refinement 

A factor which remains after the first level of refinement has an underlying 
tournament which is regular. To refine it further we look for coherent stable 
colorings on all the subtournaments obtained by removing one root (vertex) 
from the factor (tournament). 

Definition 2. Let C\, C 2 , ■ ■ ■ , CY be a level-one stable coloring for a tournament 
T, C[, C' 2 , ■ ■ ■ , CY be a level-one stable coloring for a tournament T' , we say the 
two coloring are coherent if 

— n = m and \Ci\ = \C)\, for all 1 < i < n. 

— For any arc color E and i Y J> */ ^very vertex in Ci has k E-arcs to Cj, 
then every vertex in C[ has k E-arcs to Cj. 

Definition 3. Suppose T is a regular tournament with vertices vi, ..., Sup- 
pose CY, C' 2 ^ ■ ■ ■ ) ® level-one stable coloring of T — Vi. We say that the 

collection of these level-one stable colorings constitutes a level-two stable coloring 
for T, if they are coherent with one another, and for 1 < i < n and 1 < j < iTii, 
either Vi dominates all vertices in , or Vi is dominated by all the vertices in 

cT- 

Below we describe how a regular polynomial can be manipulated algebraically 
so that either a level-two stable coloring on the underlying tournament is iden- 
tified, or the polynomial is split. 

In general let / be a polynomial of degree n with distinct roots a\, ..., a„ in 
Fp as before. Let R = Fp[x]/{f) = Fp[A\, where A = x mod f. Let f* £ R[x] so 
that 

f{x) = {x-A)f*. 

There exist uniquely determined primitive idempotents Ci £ R, 1 < i < n, 
such that J27=i In fact, Ci = ~ ~ 
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For every element c G R, these exist unique elements ci , • • • , c„ such that c = 
X^r=i canonical projection of c on Fp. The canonical 

projections of a polynomial in can be similarly defined. In particular, 

n 

/* = X! where fi = J]^(x - aj). 

i=l j^i 

We remark that since ft represents the subtournament obtained from the tour- 
nament of / by removing the root Oi, /* succinctly represents all these subtour- 
naments simultaneously. 

An element of R has the form h(A) where ft- is a polynomial over Fp of degree 
less than n. It is a zero-divisor in R iff the GCD of ft and / is not I. In other 
word as we attempt to find an inverse of h{A) in R by computing the GCD of 
h{x) and f{x), we either succeed or find a nontrivial factor of /. 

We can extend Ronyai’s algorithm to work on a polynomial over general 
completely splitting algebra over finite field. For definition of completely split- 
ting semisimple algebras over finite fields and completely splitting polynomials 
over such algebras, see [7] . Let R be completely splitting semisimple algebra of 
dimension m over Fp. Denote the uniquely determined primitive idempotents as 
€i G R, 1 < i < m. By definition we have X^r=i = 1, CiCj = CiSij, and 

R — 01<i<m-ftp. 

We may not know these idempotents at the begin of the algorithm. 

For any g(x) G R[x], if g{x) = where gi{x) G Fp[x], l<i<m 

and ei,---,em are the primitive idempotents of R over Fp. Define the score 
polynomial of g{x) by 

m 

S{g{x)) ='^S{gi{x))ei. 

i=l 

Thus S{g) succinctly represents the set of score polynomials S{gi) for i = 

In the following theorem we assume: (1) The ring operations can be carried 
out in polynomial time. (By polynomial time, we mean in time (mlogp)*^*-^^. 
) (2) Given a G R, we can determine whether a is a zero divisor, and if not, 
find its inverse in polynomial time. (3) Given an ftth non-residue in the field Fp 
{l\p— 1), if is an idempotent of algebra R, at least I distinct ftth roots 

of a can be found in time {ml log 

Theorem 4. Suppose R is ring with above properties. Let f G be completely 
splitting separable monic polynomial with degree n. There is a deterministic al- 
gorithm which in (nmlogp)^^^^ time either finds a nontrivial zero-divisor in R 
(hence a nontrivial factor of f), or computes S{f). 

Proof. Let / = ^ -R[^r], a„ = 1 be a monic polynomial, suppose 

/ = X! 

Kz<m 
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where for any 1 < z < n, fi{x) € Fp[x] splits completely over Fp into n distinct 
linear factors. 

Let A be the companion matrix of /, 

A= ^ Aid, 

l<z<m 

where Aj’s are n x n matrices over Fp. Let • • • , be the eigenvalues 

of Ai and /i 2 *\ • • • , be the corresponding characteristic vectors. 
Consider the following linear transformation 

G = I®A — A®I= ^ ei{I ® Ai) — Ci{Ai ® I) , 

l<z<m 

which acts on The vectors in L = G{R^^) must have the form 

® ® Mfc ^Ci). 

j,k,j^k i j,k,j^k i i 

It is a free module, having a basis {X)i<i<m ® ^i\j ^ k,l F j^k < rz}. 
The dimension of L is n{n — 1). 

Let H be the transformation of G on L. Let the characteristic polynomial 
of C = iL’’ be c{x) = ^^Ci{x)ei. Let a be one of roots of c{x). In case p = 
3( mod 4), c(x) should be (x — 1) * 2 ^ (x+ 1) * 2 \ that is to say, a € {1,-1}. 

Let T be the kernel of G — al as a linear map on L. The vector in T must 
have form 

* j,k,af-af(^P{F;y, 

where j3 is one of r'-th roots of a(If p = 3( mod 4), a = —1, (3 is any quadratic 
non-residue) . T is a free module over R of dimension ^ 

Let U be the transformation A^ I on T. The characteristic polynomial of U 
is score polynomial (with respect to a). 

Now we describe the algorithm to compute the score polynomial. The com- 
panion matrix of / is 




We compute I ® A and A® I, using Kronecker product of matrices. Then we 
construct linear space L = G{R^ ) by doing Gauss elimination on G = I ® 
A — A ® L In the process, we either encounter a zero divisor on R, or end 
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up with a basis for this linear space. Notice that i? is a commutative ring with 
identity, hence it has invariant dimension property. This implies that the number 
of independent generators we get should be n{n — 1). Notice that C = can 
be computed by the squaring technique. If p = 1( mod 4), we need to factor 
c{x) using Evdokimov’s algorithm. We then need to construct a basis for T, the 
kernel of C — al, and compute U, the matrix for the transformation A® I on T. 
These tasks can be done using standard linear algebra methods. In the last step, 
we calculate the characteristic polynomial of U. We either get a zero divisor in 
the process, or obtain the score polynomial. The whole process takes polynomial 
time. The theorem follows. 

With the above theorem we are in a position to extend Algorithms (1) and 
(2) in a natural way to completely splitting polynomials in where i? is a 

completely splitting algebra over Fp satisfying the conditions in Theorem 4. The 
steps in the algorithms which involve polynomial division or GCD need some 
modification due to the presence of zero divisors in R. To divide a polynomial g 
by a polynomial h over R, we need to check whether the leading coefficient of h 
is a zero divisor, and if not, to find its inverse in R. Encountering a zero divisor 
causes an early exit in the computation. 

Following [9], the GCD of two polynomials over R can be defined as follow. 
Let f{x),g{x) G with f{x) = fi{x)ei, g{x) = gi{x)ei, where 

fi{x),gi{x) G Fp[x],l < i < n. Define GCD{f,g) = J2i=iGCD{fi, gi)ei. For 
f,gGR[x], GCD{f,g) can be computed in polynomial time [9]. 

We call a polynomial g G regular if g{x) = X)r=i where gi G Fp[x] 

is regular for all i. The above discussion shows that we can extend the level-one 
refinement in a natural way to a regular polynomial g over a ring R satisfying 
the conditions in Theorem 4. We call this the level- two refinement on g. 

Theorem 5. There is a deterministic polynomial time algorithm which on input 
a regular f G Fp[x] of degree n, either finds a nontrivial factor of f, or succinctly 
finds a level-two stable coloring for the tournament of f in the sense that it 
factors f* as 

m n 

f* =Y[di where 5i = ^ G^^ ej 

i=i j=i 

where G^fi Gm constitute a level-one stable coloring Cj for the subtournament 
obtained by removing the j-th root from f and C\, form a level-two stable 

coloring for the tournament of f. Furthermore, m>2, and (after reordering if 
necessary) there isl<r <m, Y.i<i<r'^^9{9i) = Y.r+i<i<m'^^9{9i) = 

Proof. Let T be the regular tournament associated with /. Let Ot{x) = {u G T\x 
dominate u} and It(x) = {u G T\v dominates a;}. We will omit the subscripts 
when there is no risk of confusion. For a subset S of V{T), we denote the poly- 
nomial ris6s(^ ~ ■®) simply by S. Thus 

n 

/*(®) = '^{T-ai)ei. 
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If the tournament with n vertices is regular, then for any vertex x, every vertex 
in 0(x) has score (n— 1)/2 in T—x, while every vertex in I{x) has score (n— 3)/2. 
So 

n 

Sin = E (2) 

i=l 

Let’s examine what happens as we apply the level-two refinement on /*. First 
we apply Algorithm (1) on input f*. In case no early exit occurs, then from Eq. 
(2) we see that f* is factored as f^fj where 

n n 

/o = X! and fj = ^ /(oi)e/. 

i=l i=l 

Then as we apply Algorithm (2) on input {/5, //}> we may encounter a zero- 
divisor in Fp[A] and exit the refinement early with / split as result. If we success- 
fully run through the refinement without an early exit, then a level-one stable 
coloring has been found for each Ti simultaneously. Moreover, not encountering 
a zero divisor means that these level-one stable coloring are coherent ( notice that 
during the computation if we obtain a polynomial g = '^i 9 iix)ei, such that two 
of the component polynomials have different degrees, then the coefficient of the 
highest order term of g is a zero divisor.), thus a level-two stable coloring has 
been succinctly constructed in the factors of /* over R. The rest of the asser- 
tions follows from the fact that /J is of degree Finally the polynomial time 
bound follows from Theorem 4. 

A tournament is called doubly-regular if it is regular and for every vertex v, 
the subtournaments induced on 0{v) and /(u) are regular. For a doubly regular 
polynomial /, /* may go through level-two refinement being factored only into 
fo and /;. 

Define the score vector of a tournament as the sorted list of all the scores 
in the tournament. We call a tournament pseudo- vertex-symmetric if the score 
vector is the same for the subtournaments induced on 0{v) (respectively /(f)) for 
every vertex v [1]. Intuitively speaking, pseudo- vertex-symmetric tournaments 
are rare. 

From the proof of Theorem 5, we can also conclude 

Corollary 1. If f has level-two stable coloring, it corresponds to a pseudo- 
vertex-symmetric tournament. If f* has only two factors, f corresponds to 
doubly-regular tournament in Paley graph. 

The proof of Theorem 5 can be extended in a natural way to show that there 
is a deterministic polynomial time algorithm which on input a regular / G 
of degree n, where i? is a ring satisfying the conditions in Theorem 4, either finds 
a nontrivial zero divisor of R, or succinctly and simultaneously finds a level-two 
stable coloring for each canonical projection of /. 

Suppose / is regular with degree n. Let Rq = Fp, R\ = Rq[x]/ fix). Applying 
Theorem 5 to /, we either find a nontrivial factor of / or split f* over R\. 
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Suppose f* is split over Ri. Let fi be the factor with least degree ni. If rii = 1, 
then we can construct a zero-divisor in Ri [7], hence factor /. Otherwise fi is 
a polynomial whose canonical projections are regular polynomials with order 
til. Let i ?2 = Ri[x]/fi{x). Then i ?2 is a completely splitting algebra over Fp 
satisfying the conditions in Theorem 4 [7] . 

Let fi = X)r=i where R = rii<j<ni(^ ~ ^o^e that we then have 
/i = Let ^ = a; mod fi{x) and = {x - A)f*. 

The idempotents over Ri are 

^*3= n a^k^i)/ n ( Y 

k,k^j l<z<n k,k^j l<z<n l<z<n 

= ( n (^-«fcV n 

l<z<n k,k^j k,k^j 

Let = Uk.k^M ~ °'k^)/Uk.k^ji°'‘j^ - Then R 2 A canonical primitive 
idempotents are < i < n, 1 < J < ni}, and 

n ni 
i=i j=i 

We apply Algorithm (1) and (2) on over i? 2 - Either an early exit leads to 
the splitting of fi over Ri or even the splitting / over Fp; or a level-two stable 
coloring is simultaneously found on every canonical projection of /i. Inductively, 
let fi be the polynomial resulting from the z-th level of refinement with degree 
Hi over a completely splitting algebra Ri. 

A tournament is called triply-regular if it is regular and for every vertex v, the 
subtournaments induced on 0{v) and I{v) are doubly-regular. It is a remarkable 
fact that there is no triply-regular tournament with n > 4 vertices[10]. 

Theorem 6. For any polynomial f € Fp[x] of degree n, the number of levels 
that our algorithm go through in order to factor f completely is at most . 

Proof. We know that Uj+i < rii/2. If the canonical projections of fi are not 
doubly-regular, then rii+i < rii/4, and rii +2 < ni/8. Otherwise, if the projections 
of fi are doubly-regular, it is possible that rii+i = rii/2, but then the projections 
of /i+i will not be doubly-regular, since there is no nontrivial triply-regular 
tournament, hence nj _|_2 < rii+i/4, thus we have nj _|_2 < ni/8. Therefore ut < I 



4 Discussion 

In general suppose T is a tournament that admits a level- two stable coloring. Put 
two arcs uv and xy in the same class iff in the stable coloring ofT — u and T — x, 
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V and y are in corresponding classes (that is w G (7“ and y G for some j) . For 
any arc class G, we call graph Bq = {V, G) a base graph for T with respect to 
the level-two stable coloring of T. Suppose T is the underlying tournament for 
a regular polynomial / and the level-two stable coloring is represented by the 
factoring of f* into the product of g* as in the theorem above. Then the base 
graphs are in one-one correspondence with the factors g*. Each base graph is 
a regular digraph and the set of arcs in a base graph is the union of some arc 
orbits in the tournament under the automorphism group of the tournament. 

The bound on the number of levels in the above theorem seems to be far from 
being tight. Define a function j3 from set of tournaments to the set of natural 
numbers as follows. If a tournament T is not regular or doesn’t have second 
level stable coloring, (3{T) = 0. If T has second level stable coloring, (3{T) is 
the maximum of P{C,T) among all the level-two stable colorings C of T, where 
P{C, T) is the number of arcs in a minimum base graph with respect to C. 

Conjecture 1. Suppose T is a regular tournament on n vertices that admits a 
level-two stable coloring. Then for any level-two stable coloring of T, C'(’\ • • •, 
Glj^, there is a Cj, such that /3(C'j) = 0(n'^), where c < 2 is a constant indepen- 
dent of T. 

Intuitively, the coherence requirement should already make it difficult for all 
the Cj to have large minimum base graphs, if they could all have level-two stable 
colorings at all. The conjecture implies that our deterministic algorithm factor a 
polynomial of degree n over Fp completely within time logp)*^^^^. The 

fact that there is no triply-regular tournament with more than three vertices and 
the following observation of Babai [2] provide strong evidences for the conjecture. 

Proposition 1. Let T he a vertex- transitive tournament with n > 1 vertices. 
Let vq be a vertex of T. Then for every vertex v\ yf vq there exists a vertex 
V 2 vq,vi such that the size of the orbit of the pair (vi,V 2 ) in the stabilizer of 
Vo is at most {n — l)/2. 

Another way to improve our results is to look at the case when we have a lot 
of arc colors. 

Definition 4. We call a tournament with n vertices transitive if there is a linear 
order of its vertices, vi,V 2 , ■ ■ ■ ,Vn, such that for any i and color G, if Vi G- 
dominates fi+i, then Vi G -dominates Vj for any j > i. 

Denote S{p) be the size of largest transitive subgraph in a cyclotomic tourna- 
ment. Heuristically when the number of colors gets bigger, 6{p) should become 
smaller, even down to a constant. One can for example proves that a random 
tournament with n vertices and (c < 1) colors has only constant size tran- 
sitive subtournament. We can prove that the polynomial in Fp can be factored 
completely in time P{n^^^\logp), where P is a polynomial function. 
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Abstract. We discuss computation of the special values of partial zeta 
functions associated to totally real number fields. The main tool is the 
Eisenstein cocycle 'E, a group cocycle for GLn{2,)\ the special values are 
computed as periods of E, and are expressed in terms of generalized 
Dedekind sums. We conclude with some numerical examples for cubic 
and quartic fields of small discriminant. 



1 Introduction 

Let A/Q be a totally real number field of degree n with ring of integers OK^ and 
let U C be the subgroup of totally positive units. Let f, b C Ok be relatively 
prime ideals. Then the partial zeta function associated to these data is defined 
by 

Cf(b,s) :=^fV(a)-^ 

a~b 

where a ^ b means ab“^ = (a), where a is a totally positive number in 1 + fb~^. 
According to a classical result of Klingen and Siegel [10], the special values 
Cf(b,fc) are rational for nonpositive integers k. Moreover, the values Cf(b,0) are 
especially important because of their connection with the Brumer-Stark conjec- 
ture and the Leopoldt conjecture [7,6,3,8,11]. 

In [9], one of us (RS) gave a cohomological interpretation of these special 
values by showing that they can be computed in finite terms as periods of the 
Eisenstein cocycle. This is a cocycle E e Ad), where Ad is a 

certain GL„(Z)-module. Then two of us (PEG and RS) showed in [5] that the 
Eisenstein cocycle is an effectively computable object. More precisely, using the 
cocycle one can express ff(b,k) as a finite sum of generalized Dedekind sums, 
and the latter can be effectively computed by a continued-fraction algorithm 
that uses a generalization of the classical Dedekind-Rademacher reciprocity law. 

In this note we describe an ongoing project to build a database of Cf(b, 0) for 
various fields K and ideals f, b. We recall the definition of the Eisenstein cocycle 
and its relation to the special values (§2), and discuss the effective computation 
of Dedekind sums (§3). We conclude with examples of special values for some 
fields of degree 3 and 4 (§4). 
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2 Dedekind Sums and the Eisenstein Cocycle 



2.1 



Let (T be a square matrix with integral columns G Z” (j = 1, . . . , n), and let 
L C Z” be a lattice of rank r > 1. Let v € Q”, and let e G Z” with ej > 1. Then 
the Dedekind sum S associated to the data {L, a, e, v) is defined by 



S = S{L, a, e, v) := ^ e{{x, v)) 

x^L 



det a 

' 



( 1 ) 



Here {x,y) := is the usual scalar product on M”, e(t) is the character 

exp(27Tzt), and the prime next to the summation means to omit terms for which 
the denominator vanishes. The series (1) converges absolutely if all Cj > 1, but 
may only converge conditionally if Cj = 1 for some j. In this latter case we can 
define the sum by the Q -limit 



^'a(a;) 



x^L 



:= lim 

Q t^oo 






(2) 



where Q is any finite product of real-valued linear forms on M” that doesn’t 
vanish on Q” \ {0}. One can precisely determine how the value of (1) depends 
on Q ([9, Thm. 7]). The sum S is always a rational number times a power of 
27ri. 



2.2 

We recall now the definition of the Eisenstein cocycle 'I' and its relationship with 
the special values Q{b,k). For simplicity, we describe only material necessary to 
compute the special value at fc = 0, and refer to [9,5] for other k. 

Let A = (Hi, . . . , An) G (GL„(M))” be an n-tuple of matrices. For an n-tuple 
d= (c?i, . . . , dn) of integers 1 < di < n, let A{d) C M” be the subspace generated 
by all columns Aij such that j < di. (Here Aij denotes the jth column of the 
matrix Ai.) Writing A{d)-^ for the orthogonal complement of A{d) in M”, we let 

n 

X(d) = H(d)''' \ IJ CTi'-, where ai = Aidi- (3) 

The n-tuple A determines a decomposition of \ {0} into linear strata 

U (4) 

deD 



indexed by the finite set 



D = D{A) = {d I X{d) yf 0}. 
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Associated to this decomposition is a collection of rational functions "(piA) on 
M” \ { 0 }, defined by 



^p{A){x) 



det(ai, 

(a;, (Ti) • • • (a;, (J„) ’ 



if a; e X{d). 



Note that ^p{A){x) is well-defined by the construction of X{d). 

Let V € M”, and let Q be defined as in §2.1. Then the Eisenstein cocycle E 
is defined as 

<E = <E{A){Q, v) := (27Ti)"" ^ e((a;, u))'0(A)(a;) 

xez" ^ 



One can show that S' is a homogeneous (n— l)-cocycle for GL„(Z). Furthermore, 
we can express E in terms of Dedekind sums 



E{A){Q,v) 



(27tz) ’^^S{L{d),a,l,v) 
d^D 



( 5 ) 



where a is the matrix with columns (z = l,--- 5 zz), L(d) is the lattice 

A{d)-^ n Z”, and 1 is the vector (1, . . ., 1). 



2.3 

Now we describe how E can be used to compute special values. Let VF be a 
Z-basis for the fractional ideal and let W* be the dual basis 

with respect to the trace form. Via the n real embeddings Ti, i = 1, . . . ,n, any 
X & K determines a row vector {ti{x), . . . ,Tn{x)). Hence we may identify W 
with a matrix in the jth row of this matrix is the image of the jth 

basis element of W. Let 

i 3 

and let z; G Q” be defined by Vj = Tr{W*). 

Let V = n — 1, and let Si,. . .,Ei, be a basis for the totally positive units U. 
Using the regular representation p with respect to the basis W , we identify the 
units £j with elements Aj = p(£jY G GLnCE). Using the bar notation 

[All • • • [Aj/] := (1, Ai, A 1 A 2 , . . . , Ai • • • Aj/) G (GL„(Z))”, 

we have the following proposition expressing the zeta values in terms of the 
Eisenstein cocycle: 

Proposition 1 . [9,5] Let C/j he the subgroup C/ n (1 -I- f), and let n run through 
all permutations 0 / {1, ... , n}. Then 

Cf(fa,0) = ?7 ^sgn(7r)iF([A,,(i)|---|A,,(^)])(g,p(£)*z;). 

sEU/Uf 

Here 77 = ±1 is defined by 

rj = (— l)^sgn(det lU)sgn(i?), 
where R = det(log Tj(£i)), 1 < i, j < v ■ 
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3 Diagonality and Unimodularity 

3.1 

We define the rank of S = S{L, a, e,v) to be the rank of the lattice L. It is 
easy to see that after a GL„(Q) transformation, we may assume that L is the 
sublattice C Z” spanned by the first £ standard basis vectors, where £ is 
the rank of L. Furthermore, by multiplying by an appropriate rational factor, 
permuting columns and repeating columns if necessary, we may assume the pair 
{Z^, a) satisfies the following conditions: 

(i) For each column aj, the vector of the first £ components of aj is primitive 
and integral. 

(ii) If two columns of a induce proportional linear forms on Z^, then these two 
linear forms coincide on Z^, and are adjacent columns of a. 

(iii) The vector e = 1. 

Let S{Z^ , (j, 1, u) be a Dedekind sum satisfying the three conditions above. 
Let 7 t: be the projection on the first £ components, and let 7r(<T) be the 

£ X n matrix with columns 7r((Ji). 

Definition 1. Let M{a) he the set of maximal minors ofn^a). Then the index 
of S, denoted ||S'||, is defined to he 

max |detr|. 

A Dedekind sum is unimodular if US'!! = 1. 

3.2 

Now define a partition 

S 

W = LI -^ < s < n (6) 



as follows. Put 

b j G Ik if and only if 7r((Ji) = 7r((jj). 

In other words, two elements of |n] are in the same set of the partition if the 
corresponding columns of a induce the same linear form on Z^ . 

Let pfc — 

Definition 2. The vector p{S) = {pi, . . . ,ps) is called the type of S. A Dedekind 
sum is called diagonal if p{S) has length £. 
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3.3 

The virtue of diagonality is that a diagonal Dedekind sum S may be evaluated 
as a finite sum of products of generalized Bernoulli polynomials. Furthermore, 
the number of terms in this finite sum is the index of S. Hence diagonal and 
unimodular Dedekind sums can be evaluated very rapidly. 

In general, the Dedekind sums in (5) aren’t diagonal. However, we have the 
following theorem, which is the main result of [5] : 

Theorem 1. [5] Every Dedekind sum S{L,a,e,v) can he expressed as a finite 
rational linear combination of unimodular diagonal sums. If n, Rank L, and e 
are fixed, then this expression can he computed in time polynomial in log||S'||. 
Moreover, the number of terms in this expression is bounded by a polynomial in 
log||5||. 

The key ingredient in the proof of Theorem 1 is a “reciprocity law” for 
higher-dimensional Dedekind sums. For any nonzero point v G M”, let u"*" be the 
hyperplane {a: | (v, x) = 0}. Let Q be a finite product of real-valued linear forms 
on R” that do not vanish on Q” \ {0}. 

Proposition 2. Let (Tq, . . . , cr„ G Z” be nonzero. For j = 0, . . . ,n, let be the 
matrix with columns (Jq, . . . , dj, . . . , Fix a lattice L C Z”, and assume e = 1. 
Then for any v G M”, we have the following identity among Dedekind sums: 

{-iyS{Lraf,afil,v) . (7) 

Q 

We refer to [5] for proofs of the above statements. Here, in the following 
two sections, we show how Theorem 1 is applied with a rank 2 example. For 
simplicity we ignore issues of convergence, and merely remark that all of our 
manipulations with sums are compatible with the Q-limit process (2). 



'^{-iyS{L,afil,v)\ = 
j=o ^ j=o 



3.4 

Let L C Z^ be the lattice spanned by the first two standard basis vectors, and 
let 

/I 0 1 1\ 

0 12 2 

0010 ’ and u = (0,0, 0,0). 

yO 0 0 ly 

Hence S{L,a,e,v) denotes the absolutely convergent sum 



E' 



1 

xy{x + 2yfi ’ 



where the prime on the summation indicates that we omit the terms (x, y) for 
which x, y or x + 2y vanish. This sum isn’t diagonal, since a induces 3 different 
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linear forms on L instead of 2. Note also that the last two rows of a don’t affect 
the value of the sum; this observation will play an essential role when we apply 
Proposition 2 to simplify S. 

To diagonalize S, we begin with the identity of rational functions 

112 
_l_ 

xy{x + 2yY y{x + 2y)^ x(x + 2yY 

This is true provided none of the denominators vanishes. The numerators of the 
functions on the right come from the following identity of linear forms on L\ 

(w, (1, 2, *,*)*) = (w, 1 • (1, 0, *,*)* + 2 • (0, 1, *, *)*), for all w G T. (9) 



Here the stars denote entries that we don’t care about, since they don’t affect 
the value of the linear form on L. We want to sum both sides of (8) over pairs 
{x, y) G to obtain an identity among Dedekind sums of the form 



E' 



1 

xy{x + 2yY 



E' 



1 

y{x + 2yY 



+ E' 

(x,y)Gly^ 



2 

x{x + 2yY 



(10) 



However, as written (10) is incorrect. The identity (8) only holds if none of x, y, 
or X + 2y vanish, but the sums on the right of (10) include some of these terms 
(for instance, the first sum on the right of (10) contains terms (x, y) with x = 0). 
We account for this by subtracting two rank 1 Dedekind sums from the right of 
(10) as “correction terms”: 



E' 



1 

xy{x + 2yY 



E' 



E' 



y(^ + 2y)3 x{x + 2yY 

- y i V' i 

y(^ + 2y)3 x{x + 2yf 



x—0 



y=0 



E' 



1 



E' 



(x,y)GZ^ (ai,y)eZ2 



-E'— 

^ 8y4 



-Y'—. 

^ x^ 

xGZ 



( 11 ) 



Note that all of the sums on the right of (11) are now diagonal. 

This equation is precisely an instance of the reciprocity law (Proposition 2). 
To see this, apply the law with (Jq, . . . , <74 the columns of the matrix 

/I 0 1 1 1\ 

0 12 2 2 
00100 ’ 

\0 0 0 1 0 / 

and with L and v as above. Note that 174 = (Jq + 2ai, which is exactly the linear 
relation (9). The three rank 2 sums (respectively, the two rank 1 sums) are the 
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left (resp. right) of (7). All other sums vanish identically, either from the linear 
dependence among the Ui, or because all terms are meaningless. Notice how we 
used that the last two rows of the at have no effect on the sum: this enabled us to 
introduce a linear dependence among the ui that killed some of the nondiagonal 
sums. 

To diagonalize a general Dedekind sum S{L^ a, 1, v), one considers the con- 
figuration C C M” of linear subspaces consisting of (L 0 R)-*- and the spaces 
generated by the points cti, . . . , cr„. One shows by investigating the geometry of 
C that a point (Jq can be found such that when Proposition 2 is applied with 
the tuple ((Jo, . . - the resulting Dedekind sums are “closer” to diagonality 
in a certain sense. It may take several applications of Proposition 2 to express a 
Dedekind sum as a linear combination of diagonal sums. 



3.5 



The second rank 2 sum on the right of (11) has index 2. We will show how to 
make this sum unimodular. Let (Jq, . . . , (J 4 be the columns of the matrix 



/I 1 1 1 0\ 

0 2 2 2 1 
0 0 1 0 0 ’ 
\0 0 0 1 0 / 



and let L and v be as above. Then an application of Proposition 2 yields 
2 -1 1 



E' 



E' 



E' 



x(x-\-2y)^ ^ ^ x(x + 2y)^y 



+E 



. 1 



+ E T4- 



(12) 






Now all the terms on the right of (12) are diagonal and unimodular except 
for the second rank 2 sum. In fact, this sum is no longer diagonal. However, one 
further application of Proposition 2 will make this sum diagonal and unimodular. 
Hence we will have succeeded in expressing the original sum as a finite linear 
combination of diagonal, unimodular Dedekind sums. 

In proceed in general, one must be able to construct the “index-reducing” 
vector (J4 as above. An easy argument using Minkowski’s Theorem guarantees 
the existence of such a vector [1]. To construct this vector in practice, one may 
use LLL-reduction and [4, Conjecture 3.9]. 



4 Examples 

Here we present some numerical examples. For simplicity we compute C = 
Cf(b,0), where f = NOk for various rational integers TV, and b = Ok- These 
fields are the first entries in the tables of totally real fields with small discrimi- 
nant, available from [2]. 
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Cubic Gelds 

• K = Q{9), where 6*^ + 6*^ — 20 — 1 = 0 (discriminant 49). 
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• K = Q(0), where 0^ — 30 — 1 = 0 (discriminant 81). 
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• K = Q(0), where 0^ + 0^ — 30 — 1 = 0 (discriminant 148). 



N 


N-C 




N-C 


m 


N 


•c 


m 


N-C 


m 


N-C 


m 


N-C 


m 


N-C 


m 


N-C 






4 


1 


7 




2 


10 


-2 


13 


-22 


16 


7 


19 


82 


22 


68 


2 


0 


5 


-4 


8 




3 


11 


-18 


14 


-20 


17 


100 


20 


4 


23 


12 


3 


2 


6 


4 


9 


- 


10 


12 


5 


15 


42 


18 


-32 


21 


-78 


24 


23 


= 


Q{e), 


where 0^ 


- 


02 


-40- 


-1 = 0 (discriminant 169). 
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Q(0), where 0^ 
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— 40 + 3 = 0 (discriminant 257). 
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Quartic Gelds 

• K = Q(0), where 0"^ — 0^ — 30^ +0 + 1 = 0 (discriminant 725). 
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• K = Q{9), where 9'^ — — 40^ + 46* + 1 = 0 (discriminant 1125). 
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• K = Q{9), where 6*"^ — 69^ +4 = 0 (discriminant 1600). 
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6* + 1 = 0 (discriminant 1957) 
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• K = Q{9), where 6*"^ — 56*^ +5 = 0 (discriminant 2000). 
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• K = Q{9), where 6*"^ — 46*^ +2 = 0 (discriminant 2048). 
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Abstract. We explain how to construct efficiently tables of quartic fields 
by using Dirichlet series coming from Kummer theory, instead of the 
traditional methods using the geometry of numbers. 



1 Introduction 

Up to now, methods based on the geometry of numbers have played a leading 
role in the construction of tables of number fields of fixed degree and signature. 
We give a brief summary of these methods (see for example [7] for an overview 
and [9], [11], [12]). 

Let L be a number field of degree n, signature (ri, r- 2 ) with ri +2r2 = n, and 
discriminant d{L). If L is a primitive number field, a theorem of Hunter asserts 
that there exists an algebraic integer a G such that L = Q(a) and for which 
we have 



(*) 

{ii) 



0 < TrL/Q(a) < [n/2j 

Ki<n ^ ^ 



where 7 n_i is Hermite’s constant in dimension n — 1 and the OiS are the conju- 
gates of a in C. When the field L is imprimitive, we must instead use relative 
versions of this theorem due to Martinet [10]. 

Using these bounds, it is straightforward to reduce to the enumeration of a 
finite number of polynomials defining all the number fields that we are looking 
for. 

The main problem with the above method is that it requires the enumeration 
of a huge number of polynomials. For each of them we need to check that they 
are irreducible, to compute the discriminant of the field that they define and 
then to determine those which give isomorphic number fields. Thus, even for 
small degrees this method is highly inefficient. 

Constructing tables of quadratic fields poses evidently no more problem than 
detecting squarefree numbers. Thanks to the work of K. Belabas [1], similar 
constructions are available for cubic fields. We thus concentrate on the next 
case, the construction of tables of quartic fields. 
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Such tables have been constructed by Buchmann, Ford and Pohst (see [2] 
and [3]) using the method described above, and are available for example at the 
URL 

ftp : //megrez .math.u-bordeaux . f r/pub/numberf ields/degree4\kern. Bern. 
They contain all the quartic fields L with discriminant \d{L)\ < 10® for the three 
possible signatures. 

We describe here another method for constructing much more extensive ta- 
bles of quartic number fields. 

In a separate paper (see [5]), we explain how to count number fields L of 
absolute degree n having a Galois group (of the Galois closure) isomorphic to 
a given permutation group on n letters G, and discriminant d{L) bounded in 
absolute value by a given X. This is done essentially by using Kummer theory 
to compute an explicit formula for the Dirichlet series 

L/Q ' ^ 

where the sum is over isomorphism classes of number fields L having Galois group 
of the Galois closure isomorphic to G. The above paper contains implicitly the 
construction of the corresponding number fields L, and the aim of the present 
paper is to make this construction explicit. This leads to the construction of much 
more extensive tables than is possible using the geometry of numbers, and we 
could in principle construct tables which are up to 100 or even 1000 times larger 
than existing tables. We have not yet done so, essentially for storage reasons. It 
is to be noted that our methods rely in a crucial way on the fact that the group 
G be solvable, hence we would not be able to construct or ^5-extensions for 
instance. 

To illustrate our constructions, we will concentrate on the two specific exam- 
ples G = Di (dihedral group with 8 elements) and G = A^. Indeed, the Abelian 
cases G = G2, G = G3, G = G4, and G = C2 x C2 can be treated in a more 
elementary manner, the case G = S3 is better treated using Belabas’s method 
(see [1]), and the case G = 54 is very similar to the case G = A4, replacing cyclic 
cubic resolvents by noncyclic cubic resolvents in the construction we give below. 

In both the cases G = D4 and G = A4, we will need to study relative 
quadratic extensions, so we start with this first. 



2 Relative Quadratic Extensions 

Let K he & given base number field, for the moment arbitrary. We will specialize 
later to the case [AT : Q] < 3. Denote by (ri,r2) the signature of K and by 
n = ri -I- 2 r2 = [iL : Q] its absolute degree. Gonsider the following generalization 
of the Dirichlet series <?2(G2, s) as follows: 
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In the above, the sum is over quadratic extensions of K up to K -isomorphism, 
necessarily with Galois group isomorphic to C 2 , X){L/K) is the relative discrim- 
inant ideal of Lj K, and finally Af denotes the absolute norm from K to Q. 

One of the surprising results of the study made in [4] is that there is a very 
simple expression for this Dirichlet series, and consequently for the number of 
quadratic extensions of K of bounded discriminant. Before giving the result, we 
need a definition. 

Definition 1. Let c be an integral ideal of K such that c | 21k- 

(1) We define the Selmer group modulo by 

c rm\ ^ 3 ;^ = u (mod *c^)} 

Or2[K ) = . 

{u € K* / {u, c) = 1} 

When c = Ik we will simply speak of the Selmer group of K. 

(2) Let ao be given such that ao = 1 (mod *c^). We will denote by T^ 2 [aQ) the 
set ofuG S^ 2 [K) such that for any ideal Ci different from c and coprime to 
ao such that c | Ci | 21k, there is no solution to the congruence = aou 
(mod *c^) (it is understood in this definition that u is lifted to an element 
coprime to Ci and not only to c, which is always possible). 

The following results follows immediately from the results of [4] . 

Theorem 1. There exists a bisection between quadratic extensions L of K up to 
K -isomorphism (together with the trivial extension K/K), and the set of triples 
(c, a,u) where c is an integral ideal dividing 21k, ci is an integral squarefree ideal 
coprime to c such that the class of a belongs to the square of an ideal class in the 
ray class group Cfi 2 [K), and u G T„ 2 (ao), where ao = 1 (mod *c^) is such that 
aq^ = aolK for some ideal q. With this notation, the extension corresponding 
to the triple (c, a, u) is L = K{^/afiu) and the relative discriminant t:{L/K) is 
equal to 4a/c^. 

From the above, it is easy to obtain the following corollary. 

Corollary 1. With the above notation, we have 



<1>2,k{C 2, s) = -l + n ( 1 - 



E 

a&Cl^2(Kfi 

a squarefree 



1 

Afa® 



Furthermore, in [4] , the following easy result is proved: 
Proposition 1. We have a canonical exact sequence 



3(2 (K) — > Szk(K) 



{Zk/c^Y 



CI(2{K) 

cI(2{ky 



From this proposition, we immediately deduce the following corollary: 
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Corollary 2. 



\SAK)\ 



2^^+^^\Cl,2{K)/Cl,2{Kf\ 

Afc 



Putting all this together, a small computation gives (see again [4]) 



Theorem 2. Let K he a number field of degree n and signature (ri,r 2 ). 



(1) We have 



^ 2 ,if(C' 2 , s) = — 1 + 



1 

2„(2s-l)+r2^^(2s) 



c|2 X 



where as usual fx denotes the Dedekind zeta function of K, Cif(X)S) = 
rip(l-x(p)A/'p)“ is the L-function attached to the character \ (which we 
denote in this way so as not to confuse it with the ordinary Dirichlet L- 
series which will also occur), and in the inner sum \ runs over the quadratic 
characters of the ray class group Cfi 2 (K) corresponding to the modulus c^. 

(2) The number N 2 ,k{C 2 , X) of quadratic extensions L of K up to K-isomor- 
phism such that Af(ii{L/K)) < X satisfies 



N2,k{C2,X) 



1 Res^^iCg(s) _ 
2-2 ^^( 2 ) 



We will not need the formula for N 2 ^k{C 2 , X) in the present paper, but its 
simplicity is remarkable, so it deserves to be better known. Although we have 
proved it using our methods, it can be found slightly hidden in the well known 
paper of Datskowsky and Wright [8] on relative cubic extensions. 

We will directly use Theorem 1 to find all quadratic extensions LfK such 
that Af{d{L/K)) = N, as follows. 

1) Make the list of all ideals c dividing 2 such that 4”/Afc^ = Nifl/cfi 
divides N . For each such ideal c, execute the following steps. 

2) Compute the elements of S „2 (K), and make a list of all integral squarefree 
ideals a which are coprime to c and such that Afa = N/ Af{2/cfi. 

3) For each of the ideals a in the list, test whether the class of a is a square 
in the ray class group Cfi 2 {K). If it is, compute an element ao such that aq^ = 
aoZx with oq = 1 (mod *c^). 

4) For each suitable ideal a, compute as given in Definition 1 as a 

subset of Sc 2 {K). 

5) The set of extensions L/K such that Af(b{L/K)) = fV up to A'-isomor- 
phism is given by L = K{y/oifiu) where ao is as above, and u € T,, 2 (ao), except 
that when a = Zx (in which case one may take oo = 1)? we must exclude u = 1 
to avoid the trivial extension. 

Let us consider an explicit numerical example of the above theorem. Let 
K = Q(-\/^) be the base field, so that T 2 = 1 and n = 2. The prime 2 is inert 
in K, hence the only ideals c are c = Zx and c = 2Zx- The ordinary class 
group is of course trivial, and an easy computation shows that the ray class 
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group CUziiiK) is of order 2, generated by the class of the ideal The 

only nontrivial character of this ray class group is easily seen to be given by 
x(a) = (;^)- A short computation gives for K = Q(-\/^): 



s) = — 1 + 



1 

2Ck{2s) 




Ck{s) + Ck 





In addition, we know of course that Ck{s) = C(s)T ((— )) s)- 

A small program shows immediately that the first 50 terms of the above 
Dirichlet series are given by 



^2,if(C'2, s) 



2 12 12 2 1 



This means that, up to iC-isomorphism (and not up to Q-isomorphism), there 
are two quadratic extensions LjK such that N{p{LlK)) = 13, one such that 
N{p{L ! K)) = 16, and so on. We have limited to 50 terms for ease of presentation, 
but on a computer there is no difficulty in obtaining 10^ terms for example. 

Thus, we know that there are exactly 2 quadratic extensions LjK such that 
N{t}{LlK)) = 13, and we want to compute them explicitly using the above 
algorithm. 

In step 1, we must take c = 2Zk, otherwise A/"(2/c)^ is even. In step 2, the list 
of ideals a such that N a = 13 is evidently given by the two prime ideals above 
13, generated by (7 + ^/^)/2 and (7 — -y/^)/2 respectively. An immediate 
computation shows that the class of both these ideals belongs to Ct^2, with 
ao = — 1 — 2-y/^ and oq = — 1 + 2 y/^ respectively. 

Finally, we compute in a naive way that = {Al}) &nd since —1 is not a 
square modulo 4 (in Zk), we deduce that the sets S^2[K) and T^2{ao) are equal 
to {1}. Finally, the desired extensions are the two extensions iC(\/— 1 ± 2\/^^). 
Considered as extensions of Q, these two extensions are Q-isomorphic to the 
unique quartic field of discriminant 117 = 3^ • 13, with Galois group isomorphic 
to D 4 , of which an absolute equation is for example — X"^ + X + 1 = 0. 

Let us do the same computations for Af{d{L/K)) = 16, for which the Dirichlet 
series tells us that there is only one extension. Since N = 16, all possible c 
dividing 2 may be possible, and since 2 is inert in K, we must look at c = Zk 
and c = 2Zk- 

For c = Zk, we must list ideals of norm 1, hence the only possible ideal a is 
a = Zk, and we can evidently choose oq = 1- Since S'z^(iL) = {±1}, since —1 
is not a square modulo ^Zk and since 1 is excluded in the special case a = Zk, 
we obtain the extension L = Considered as an absolute extension of 

Q, this is isomorphic to the unique quartic field of discriminant 144 = 3^-16 
with Galois group isomorphic to (72 x (72, of which an absolute equation is for 
example X'^ — X'^ + 1 = 0. 

For c = 2 Zk, since 2 is inert in K, the only ideal of norm 16 is the ideal 
a = AZk which is not squarefree, hence no extensions are obtained in this case. 
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Let us now consider a slightly less easy example. We take K = Q(-\/— 15). 
This field has class number 2, and in addition the prime 2 is split in K, so the 
situation is slightly more complicated. We first compute the series <^ 2 ,ic(C' 2 , s)- 
For c = Z/c, in addition to the trivial character xo, we have the genus char- 
acter xi which can be defined by the formula 

Indeed, it is immediately checked that this is indeed a character on the ordinary 
class group (it is multiplicative and trivial on principal ideals) and it is nontrivial 
since it is equal to —1 on the prime ideal above 5, which is of norm 5. Thus, for 
c = Zk the quadratic characters of Cl ^2 are X = Xo and x = Xi- 

For c equal to one of the prime ideals above 2, it is immediately computed 
that the ray class group is isomorphic to the ordinary class group, so we have 
the same two characters, except that we must take care that the characters are 
0 on c. 

For c = 2Zjy, the ray class group is isomorphic to hence is generated 

by the character xi and by another character X 2 It is easily seen that we may 
choose X 2 defined by 




Indeed, this is well defined on the ray class group modulo 4 since, if a = aZ^ 
with a = 1 (mod 4) we have Na = N{a) = I (mod 4), and it is nontrivial since 
it is equal to —1 on the prime ideal above 3. Thus, the quadratic characters of 
the ray class group modulo 4 are X = XOj Xi> X 2 and XiX 2 - 
Summing up, we obtain 



^2,k{C2, s) = — 1 -I- 



1 



Ck 



2Ck{2s) 

2 

/-4 



1 - 



22s 23« 24^ 



Ck{s) 



1 2 5 4 4 _ , 

+ Ck{xi,s) 



VM-) 



Ck 



-4 

W) 



Xi,s 



A small program shows immediately that the first 50 terms of this Dirichlet 
series are given by 

^ 1 2 4 4 2 

^2 7c(C^25 s) — “t“ “t“ “t“ “t“ “t“ * * * 



Let us construct the first few corresponding quadratic extensions. First, we 
compute the sets Denote by p and p the two prime ideals above 2 in K. 

The possible ideals c are c = Z/y, p, p and 2Zjy. Since 3 is ramified in K and 
Szk {K) is of order 4 by Corollary 2, we have 



Sik{K) = {±1, ±3} , 
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and since —3 = 1 (mod 4) we have 

S,.{K) = S-,.{K) = S4z^{K) = {1,-3} . 

The quadratic extension corresponding to the norm TV = 1 is of course the 
Hilbert class field of K, clearly obtained with c = 2'Ek, ci = 'Em and u = —3 
(since u = 1 would give the trivial extension), hence is K(\/—3), which is well 
known. 

Let us now construct the 2 non-iV-isomorphic quadratic extensions Lj K such 
that N = M{^{LlK)) = 16. For c = we must take a = Ek, hence oq = 1, 
and hence Tz^ioio) = {~1)3|, so we obtain the extensions L = and 

L = K{V3).For c^p, c = p, or c = 2Ek, it is immediately seen that all integral 
ideals a having a suitable norm are either not squarefree or not coprime to c, so 
there are no other extensions, as predicted by the Dirichlet series. 

To finish this example, we construct the 4 non-iV-isomorphic quadratic ex- 
tensions Lj K such that N = j K)) = 24. Since 16 { TV, we can take c = p, 

c = p and c = 2Ek- Consider first c = p. We must find the list of squarefree 
ideals prime to c of norm 6, and clearly there is only one such ideal, the ideal 
a = pp 3 , where p 3 denotes the unique prime ideal above 3. We check immediately 
that a belongs to the square of an ideal class in Cl^- 2 {K) and that we can take 
ao = (3-1--/— 15) /2 (or its conjugate, depending on the specific choice of p). The 
only possible ideal Ci that we have to check is Ci = 2Ek, which is not coprime to 

Qfo- Thus T^ 2 {ao) = {1, —3} giving the two extensions L = K{\J (3 -I- ■/— 15)/2) 

and L = (—9 — 3-\/^T5)/2). The choice c = p would give the two other 

extensions L = K{^ (3 — ■/— 15)/2) and L = K {^ (—9 -I- 3-\/^T5)/2). Finally, 
for c = 2Ek we would need an ideal of norm 24, which would not be coprime to 
c, so no extensions are obtained in this case. 

In a similar manner, we can easily consider examples where 2 is ramified and 
the class group is nontrivial (for example K = Q(-/— 20)), or other base fields 
K such as cyclic cubic fields. Some new phenomena appear in these cases, but 
nothing essential, and we leave this as practice for the reader. 

To finish this section, we note that even though in practice it is very easy to 
compute the groups S^ 2 [K) by simple enumeration of the suitable elements of 
(K), the Selmer group of K which is generated by the units and the virtual 
units (see [7]), there is a completely algorithmic way of doing it by using directly 
the exact sequence of Proposition 1. 

3 Constructing C> 4 -Extensions 

The first application of the above results and constructions is to the computation 
of (extensive) tables of quartic H 4 -extensions of Q. We have for example com- 
puted that the number of such extensions of absolute discriminant less than or 
equal to 10^^ is equal to 5232538688240. The computation took less than 4 days 
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of CPU time. It is evidently out of the question to make a table of the corre- 
sponding polynomials, and in practice we have constructed such a table only up 
to 10^, because of storage and not of time considerations. We could if necessary 
construct a table up to 10® with approximately 3GB of storage and 48 hours 
of CPU time. This is 1000 times further than the published tables obtained by 
the geometry of numbers. With present day computers, it is quite plausible that 
using the geometry of numbers, we could construct tables which go 10 times 
further than before, but it seems unlikely that we can go 1000 times further, and 
certainly not in 48 hours of CPU time. 

To construct ^4-extensions L/Q, we note first that such extensions are im- 
primitive, in other words that there exists a quadratic field K such that K G L, 
so that L is a quadratic extension of K. Furthermore, it is clear that if r is 
the unique nontrivial element of the Galois group of K/Q then L and t{L) are 
non-it'-isomorphic but Q-isomorphic quadratic extensions of K. Finally, note the 
formula \d{L)\ = d{K)'^ Af{d{L/K)). 

Of course, not all imprimitive quartic extensions of Q are T)4-extensions. They 
can also be C'4-extensions or C 2 x C'2-extensions. However these extensions are 
much rarer (there are approximately 0.0523 X T)4-extensions of absolute discrim- 
inant up to X, compared to 0.122 C'4-extensions and 0.00275 log^ X 
C 2 X C2-extensions, see [5]), are easy to detect (for example, C 2 x C 2 exten- 
sions are the only imprimitive quartic fields with square discriminant) and are 
much easier to construct. Thus, constructing tables of ^4-extensions is essen- 
tially equivalent to constructing tables of imprimitive quartic fields. 

To construct a table of T)4-extensions of absolute discriminant up to X, we 
thus proceed as follows. 

1) First construct a table of quadratic fields K such that \d{K)\ < \fx. This 
essentially amounts to finding squarefree numbers, and in addition since \fX 
will be small (31622 for X = 10®), this computation is immediate. For each such 
quadratic field, compute the class group, the ideals c dividing 2, the ray class 
groups CIc 2 {K) and the groups S^ 2 {K) (of course this is not stored, but done 
on the fly as we go through each quadratic field in sequence). 

2) For each quadratic field K, use the techniques explained in Section 2 
to construct all quadratic extensions L of K such that N{X){L/K)) < B = 
[X/d{K)‘^\. More precisely, compute the list £ of all integral squarefree ideals 
a of norm less than or equal to B whose class is a square in the ordinary class 
group. For each ideal c | 2, extract from £ those ideals a coprime to c whose 
class is a square in the ray class group modulo c® (this is easily done once we 
know that the class of a is a square in the ordinary class group) of norm less 
than or equal to H/ A/"(2/c)®. For each such ideal a compute oq = 1 (mod *c®) 
such that aq® = ao'^K for some ideal q. The desired extensions are K{y/aou) for 
u G T^ 2 (ao), excluding the trivial extension. 

3) To avoid Q-isomorphic fields, in the above construction we identify a 
triple (c, a, u) with its conjugate by the nontrivial Galois automorphism of K. 
To avoid C 2 x (72-extensions, we exclude ideals a of square norm, and to avoid C 4 - 
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extensions, we exclude ideals a of norm equal to a square times the discriminant 
of K (this occurs only when K is a real quadratic field) . 

4 Constructing A 4 -Extensions 

The situation is considerably more complicated in this case, but it is still possible 
to construct extensive tables of ^ 4 -extensions. In fact, since according to [4] the 
number of such extensions with absolute discriminant up to X is approximately 
equal to 0.0179 log X, it is possible to go much further than the typical 
bound 10® we gave for ^ 4 -extensions. For example, we have computed that the 
number of such extensions up to 10^^ is equal to 104766 in less than 24 hours of 
CPU time, and using the method given in this section, it would be easy to build 
the corresponding table in essentially the same amount of time. 

Let L/Q be a quartic extension, let N be its Galois closure in C, and assume 
that the Galois group of N/Q is isomorphic to A4. The field N has a unique 
cubic subfield K3 which is cyclic over Q. The extension N/K3 is a biquadratic 
extension, hence contains three quadratic subextensions Li/K^ for 0 < z < 2, 
and these subextensions have trivial norm, in other words Li = with 

AfK3/q{oii) a square of Q. Finally the discriminant of L is given by d{L) = 
d{K3)J\fK3/q{'H{Li/K3)) for any z. 

Denote by ct a generator of Gal(fV/L), which can also be considered as a 
generator of Gal(iL 3 /Q). We can set a = ao and take Oj = (J*(a). If 0 is a 
square root of a in fV, we have L = Q{rj) with 

r] = 9 + a{9) + cr®(6») = . 

Explicitly, if + aX"^ + bX + c is the minimal polynomial of a, a defining 
equation for L/Q is given for example by the polynomial 

Pa{X) =X^~ 2(a® - 2b)X^ + 8cX + (a^ - 4a^b + Sac) 

whose discriminant is equal to 2^®(c — o6)® times that of the polynomial X^ + 
aX'^ + bX + c. 

Enumerating cyclic cubic fields is very easy, so there remains to explain how 
to enumerate quadratic extensions Li/K^ with trivial norm. This is done by 
generalizing the definitions and results given in Section 2 when there are no 
restrictions. 

Definition 2. Let K be a number field, c an ideal of K dividing 2, and denote 
by Af the norm from K to Q. 

(1) We define the square ray class group modulo c® by 

^ jg/ (g, c) = Zk, Afja) square} 

' {fUjKl P = 1 (mod *c®), Af{P) square} 

When c = "Lk, we will simply speak of the square class group. 
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(2) We define the square Selmer group modulo by 

S (2 [A/] = {u G S (2 (K ) / Af{u) square} . 

(3) Let ao be given such that ao = 1 (mod *c^) and Af{ao) a square. We will 
denote by Tc 2 (ao)[A/] the set ofu G S'c 2 [A/] such that for any ideal Ci different 
from c and coprime to ao such that c | Ci | 2Z/y, there is no solution to the 
congruence = uqu (mod *c^). In other words, 

Tc2(ao)[A^ = r'c2(ao) n S'c2[7V] . 



The analog of Theorem 1 is then as follows. 

Theorem 3. Let K be a number field. There exists a bijection between quadratic 
extensions L of K with trivial norm up to K -isomorphism (together with the 
trivial extension K/K), and the set of triples (c, a, u) where c is an integral ideal 
dividing 2Z/y, a is an integral squarefree ideal coprime to c which is of square 
norm, and such that the class of a belongs to the square of an ideal class in the 
square ray class group Cfi 2 [J\J\, and u G T„ 2 (ao)[N\, where ao = 1 (mod *c^) of 
square norm is such that aq^ = ao'^K for some ideal q. With this notation, the 
extension corresponding to the triple (c, a, u) is L = K{^aou) and the relative 
discriminant d{L/K) is equal to 4a/c^. 

From this theorem it is easy to deduce the following analog of Corollary 1. 

Corollary 3. Denote by <? 2 ,k(C' 2 , s)[A/] the Dirichlet series which is the gener- 
ating function of quadratic extensions of square norm. Then 



^2,k{C2, s)[A/] = — 1 + 



22n 



■^Afc2l5,2[A/]in(l- 



c|2 



ph 



Afp 



2s 



1 

, Afa® 
aeC/„2 [J\Pf 

a squarefree 




Note that in the above sum the ideals a are of square norm, hence the Dirich- 
let series <^ 2 ,ic(C' 2 , s)[A/] is a Dirichlet series in the variable 2s. 

The analog of Corollary 2 is the following, in the case we are interested in 
(see [4] for the general case). 

Proposition 2. Let be a cyclic cubic field, and set x{c) = 2 if c — 2 Zks, 
x{c) = 1 otherwise. Then 

= Wc ■ 

Putting everything together, a small computation gives (see [4]) 

Theorem 4. Let be a cyclic cubic field. Then 

= -1 + n (i - i) 

c|2 p|c ^ ^ 

xeC/,2[Ni7H,2[A/]" 
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where 



= JJ" X(Plp2) + X(Plp3) + X(p2p3) \ 

pZk3=PiP2P3 V J 

By summing over all cyclic cubic fields, we obtain 
Corollary 4. With the notation of the above theorem, we have 



MM.s) = \Y. 



1 



3 ^ fiK^Y^ 

Ks/Q •’ ^ 



’^2,K3{C2, s)[Af| 



where the sum is over all isomorphism classes of cyclic cubic fields K^/ 



Here, we simply want to use Theorem 3 for the construction of tables of 
^ 4 -extensions. As already noted, essentially the same method leads to the con- 
struction of tables of 54-extensions, simply by replacing cyclic cubic fields by 
noncyclic ones. 

To construct a table of quartic A 4 -extensions of Q of discriminant up to X, 
we thus proceed as follows. 

1) First construct a table of cyclic cubic fields such that |/(AT 3 )| < y/X, 
where /(ATs) = d{Kfi) is the conductor of K^. This is very easily done since the 
structure of set of isomorphism classes of cyclic cubic fields is completely known 
(see for example [6]), and is very fast since we need to consider conductors 
only up to the square root of X. Then for each such cyclic cubic field, compute 
the square class group, the ideals c dividing 2, the square ray class groups and 
the square Selmer groups modulo c^. Compute also explicitly the action of a 
generator a of the Galois group Gal(A' 3 /Q) on elements and ideals of K^, (see 
[6]). Then for each such perform the following steps. 

2) Make a list of squarefree integral ideals b of norm less than or equal to 

B = X/ f{K^Y = yJX/d{Kfi^ such that b is divisible only by prime ideals 

above primes of Q which are split in K^, and such that b is not divisible by 
two distinct prime ideals above the same split prime of Q. Let C be the list of 
ideals a of the form a = b(j(b) (where b ranges in the list that we have just 
found) and whose class is a square in the square class group. For each ideal c | 2, 
extract from £ those ideals a coprime to c whose class is a square in the square 
class group modulo (once again, this is now easily done). For each such ideal 
a compute ao = 1 (mod *c^) of square norm such that aq^ = for some 

ideal q. 

3) The quadratic extensions Li of trivial norm of with N{Li/Kfi) < 
X/d{Kfi) are the Kfiy/afiu) for u G Tc 2 {ao)[N], excluding the trivial extension. 
To avoid Q-isomorphic fields, in the above construction we must identify a triple 
(c, a,u) with its two Galois conjugates by a and a^. The corresponding quartic 
A 4 -extensions L of Q are obtained by computing the equation of the cubic poly- 
nomial satisfied by agu over Q and applying the formula given above for the 
corresponding quartic. 
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Abstract. For each permutation group G on n letters with n < 4, we 
give results, conjectures and numerical computations on discriminants of 
number fields L of degree n over Q such that the Galois group of the 
Galois closure of L is isomorphic to G. 



1 Introduction 

The aim of this paper is to regroup results and conjectures on discriminant 
counts of number fields of degree less than or equal to 4, from a theoretical, 
practical, and numerical point of view. Proofs will be given in a forthcoming 
paper. 

We only consider absolute number fields, and for simplicity we do not distin- 
guish between different signatures, although this can easily be done. We denote 
by G the Galois group of the Galois closure. 

If G is a permutation group on n letters, we write 

^) = E A„(G, A) = ^ 1 , 

where in both cases the summation is over isomorphism classes of number fields 
L of degree n over Q such that the Galois group of the Galois closure of L is 
isomorphic to G and d{L) denotes the absolute discriminant of L. 

Important Remark. Gertain authors, in particular Datskowsky, Wright and 
Yukie (see [7], [14], [15]) count number fields in a fixed algebraic closure of Q, 
which is perhaps more natural. This is the same as fV„(G, A) when G is of 
cardinality equal to n, i.e., when the extensions L are Galois. Otherwise, in 
the range of our study {n < 4), their count is equal to m(G)A„(G, A), where 
m(S'3) = 3, m{D4) = 2, m(A4) = m{S4) = 4. 

For each group G, we give the results in the following form: we first give ex- 
pressions for <Pn{G,s) which are as explicit as possible. Then we give an asymp- 
totic formula for A„(G, A) which is usually directly deduced from the formula 
for <Pn{G,s). In some cases the asymptotic formula can be refined, but usually 
only conjecturally. We then explain the method that we have used to compute 
A„(G, A) exactly. Finally, we give a table of A„(G, 10^) for increasing values of 
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k as well as a comparison of this data with the most refined result or conjecture 
on the asymptotic behavior of -/V„(G, X). The upper bound chosen for k depends 
on the time and space necessary to compute the data: we should not need more 
than one week of CPU time and 1GB of RAM. 

It should be stressed that although we only give the number fV„(G, A) of suit- 
able fields, the same methods can also be used to compute explicitly a defining 
equation for these number fields, but the storage problem makes this impractical 
for more than a few million fields. 



2 Degree 2 Fields with G C 2 



2.1 Dirichlet Series and Asymptotic Formulas 



Using the characterization of a fundamental quadratic discriminant, it is easy to 
show that 



^^2(G2, s) 




- 1 





C(^) . 

C(2s) 



From the above formulas, we easily deduce a crude form of the asymptotic for- 
mula: 



A2(G2, a) 
c(G2) 



c(G 2 ) a with 

0.607927101854026628663276779 . . . 

C(2) 7t2 



It is known that we have the more precise result 

A 2 (G 2 , A) = c(G 2 ) a + 0(a 1/2 exp(-c log A^/^ log log A-^/^) 



for some positive constant c, and under the Riemann Hypothesis, that 
A2(G2, A) = c(G2) a + 0(A“) 

for any a > 8/25 (see for example [13], Notes du Chapitre 1.3). It is conjectured, 
and this is strongly confirmed by the tables, that we can take any a > 1/4 in 
the error term. 



2.2 Numerical Computation 

Since l/^(2s) = X)m>i deduce from the formula for 

^ 2 (G 2 , s) the formula 



A2(G2, A) = -1 -I- ^ fi{m) 



X + rri^ 
2m? 



-k2 



A 

4m^ 



+ 2 ^ ^i{m) + ^ ^i{m) . 

^X/4<m<^X/3 yA73<m<VA 
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This is the formula that we have used in exact computations. Note that, although 
we could use directly the Dirichlet series 'p 2 {C 2 ,s), this would be much less 
efficient. 



2.3 Table 

In this table, we let P 2 {C 2 ,X) = [c(C' 2 ) X~\ be the predicted value and £ 2 ( 02 , -’f) 
= (N 2 {C 2 ,X) — P 2 {C 2 , X))/X^/'^ rounded to 5 decimals. 



A 


^ 2 ( 6 * 2 , X) 


P2{C2,X) 


E2{C2,X) 


10 ^ 


6 


6 


0 


1 Q 2 


61 


61 


0 


10 ^ 


607 


608 


-0.17783 


10 ^ 


6086 


6079 


0.70000 


10 ^ 


60786 


60793 


-0.39364 


10 ® 


607925 


607927 


-0.06325 


10 ^ 


6079285 


6079271 


0.24896 


10 ® 


60792709 


60792710 


- 0.01000 


10 ® 


607927069 


607927102 


-0.18557 


lOlo 


6079270822 


6079271019 


0.62297 


loll 


60792710200 


60792710185 


0.02667 


1012 


607927101751 


607927101854 


-0.10300 


1013 


6079271018463 


6079271018540 


-0.04330 


1013 


60792710186342 


60792710185403 


0.29694 


lOi® 


607927101852652 


607927101854027 


-0.24451 


lOi® 


6079271018544414 


6079271018540266 


0.41480 


1017 


60792710185393816 


60792710185402663 


-0.49750 


lOi® 


607927101854026495 


607927101854026629 


-0.00424 


lOi® 


6079271018540242468 


6079271018540266287 


-0.42357 



A notable feature of this table, common to most of the tables that we give, is 
the changes in sign of the error term, showing that there is no systematic bias. 
Thus, only the order of magnitude of the error term can be questioned, but the 
existence of an additional main term seems unlikely. 



3 Degree 3 Fields with G ~ C 3 

3.1 Dirichlet Series and Asymptotic Formulas 

From the characterization of discriminants of cyclic cubic fields (see for example 
[5], Section 6.4.2), it is easy to show that 
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From the above, we deduce a crude form of the asymptotic formula: 



N3{Cs,X) 



c{C3) 



c(C'3) with 

llv^ 



367T 



n 



1 - 



p{p + 1) 



p= 1 (mod 6) 

0.1585282583961420602835078203575 . . . 



It is easy to refine this to the more precise result 

NsiCs, X) = 4 C 3 ) ^1/2 + 0(X“) 

for a = 1/3, and probably with some effort for some a < 1/3. 

In view of the positions of the poles of the function <^ 3 ( 03 , s) it is reasonable 
to conjecture that we can take a = 1/6 + £ for all £ > 0. 



3.2 Numerical Computation 

From the explicit formula for <?3(C3, s) or equivalently from the explicit descrip- 
tion of cyclic cubic fields, it is easy to deduce the formula 

N3(C3,X) = -i+3 ^ /6(m)2-(-)-i+ ^ ^ 

l<m<^/X/9 V77/9<m<V77 

where feim) = 1 if m is equal to a squarefree product of primes congruent to 
1 modulo 6, and to 0 otherwise, and oj{m) is the usual function counting the 
number of prime divisors of m. 

This is the formula that we have used in exact computations. As for the C 2 
case, it would be much less efficient to use directly the Dirichlet series. 



3.3 Table 

In this table, we let P 3 {C 3 , X) = [£((73) be the predicted value and 

£ 3 ( 03 , X) = (7V3(C'3, A) — P3(C3, A))/A^/® rounded to 5 decimals. 
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A 


Al3(C3,A) 


P3{C3,X) 


E3{C3,X) 


10^ 


0 


1 


-0.68129 


10^ 


2 


2 


0 


10^ 


5 


5 


0 


10^ 


16 


16 


0 


10® 


51 


50 


0.14678 


10® 


159 


159 


0 


10^ 


501 


501 


0 


10® 


1592 


1585 


0.32491 


10® 


5008 


5013 


-0.15811 


lOi® 


15851 


15853 


-0.04309 


loll 


50152 


50131 


0.30824 


1012 


158542 


158528 


0.14000 


1013 


501306 


501310 


-0.02725 


1013 


1585249 


1585283 


-0.15781 


lOi® 


5013206 


5013104 


0.32255 


lOi® 


15852618 


15852826 


-0.44812 


1017 


50131008 


50131037 


-0.04257 


lOi® 


158528150 


158528258 


-0.10800 


1019 


501309943 


501310370 


-0.29091 



4 Degree 3 Fields with G Ss ~ 

4.1 Dirichlet Series and Asymptotic Formulas 

In this case, we may use methods coming from Kummer theory to compute the 
Dirichlet series and to deduce an asymptotic estimate for N^{S 3 ,X), but the 
results are too complicated to state here. 

On the other hand, we have the celebrated Davenport-Heilbronn theorem 
(see [8], [9] and [6], Chapter 8) which asserts that 

N^{Sz,X) ~ c{Sz) X with 

c(53) = — ^ = 0.27730245752690248956104209294 . . . 

3C(3) 

From the work of K. Belabas (see [1]), it is known that we can refine this 
estimate to 

^ 3 ( 53 , X) = c{Si) X + 0{X exp(-ci/log A log log A)) 
for any c < 1/24. 

However, much more is conjectured to be true. From the work of Shintani 
and heuristics of D. Roberts (see [11], [12], [10], where the constant ^(2) must be 
omitted), it is believed that there is an additional main term and that we have 
in fact 

^ 3 ( 53 , X) = c(53) a + c'(53) a5/6 - + o(a1/2) 
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with 



^ 3(3+73)r(l/3)3C(l/3) 

107t3 C(5/3) 

= -0.40348363666394679863364025671534. . . 



4.2 Numerical Computation 

We refer to the work of K. Belabas ([2] and [6], Chapter 8) for the use of the 
Davenport-Heilbronn method to compute N 3 {S 3 ,X). The details would be too 
long to state here. We have simply copied Belabas’s results. Note that to obtain 
the table below from his, one must first add the contributions of the complex and 
totally real cubic fields, since we do not distinguish between different signatures, 
and then subtract the contribution of cyclic cubic fields given in the table above. 

We could also use the approach based on Kummer theory. This would cer- 
tainly be much less efficient since Belabas’s method gives cubic fields in es- 
sentially linear time. The most serious obstruction would not be so much the 
complexity of the formula or the ray class groups which occur, but the fact that 
we must sum over all quadratic discriminants up to X. 



4.3 Table 



In this table, we let P 3 {S 3 ,X) = 



c{S3)X + c'{S3)X^/^-^X^/^ 



be the 



predicted value using the refined heuristics, and £ 3 ( 83 , X) = {N 3 {S 3 ,X) — 
P 3 {S 3 ,X))/X^/"^ rounded to 5 decimals. 



X 


N 3 {S 3 ,X) 


P 3 {S 3 ,X) 


E 3 {S 3 .X) 


10^ 


0 


0 


0 


10^ 


7 


8 


-0.10000 


1Q3 


149 


148 


0.03162 


10^ 


1886 


1898 


-0.12000 


10® 


21794 


21791 


0.00949 


10® 


236858 


236901 


-0.04300 


10^ 


2497935 


2497967 


-0.01012 


10® 


25855883 


25856912 


-0.10290 


10® 


264539133 


264541514 


-0.07529 


IQi® 


2686092328 


2686091377 


0.00951 


IQii 


27138004413 


27137996056 


0.02643 



5 Degree 4 Fields with G ~ C 4 





Counting Discriminants of Number Fields of Degree up to Four 



275 



5.1 Dirichlet Series and Asymptotic Formulas 

By studying discriminants of cyclic quartic extensions, it is not difficult to show 
that 



^4(C*4, s) 





From the above formula, we can easily deduce a crude form of the asymptotic 
formula: 

A4(C4, X) ~ c(C'4) with 



c{C4) 



7t 2((^'^24) n (1+ 3/2+ 1/2) 

^ ^ ^ p=l (mod 4) ^ ^ / 

0.12205267325139676092260805289654 . . . 



- 1 



It is easy to refine this to the more precise result 

A4(C'4, X) = c{C4) A+2 + 0(A“) 

for any a > 1/3. 

In view of the positions of the poles of the function <p 4 {C 4 ,s), as for the case 
G ~ 53 , it should be easy to prove that there is an additional main term, and it 
is reasonable to conjecture that in fact 

A4(G4, X) = c(G4) A+2 + c'(G4) A+3 + 0(A“) 



for any a > 1/5, with 



c'(G4) 



3 + 2-'/» + 2-"/» <(2/3) ^ f 2 \ 

1 + 2-W 4,C(4/3) +p + p>/3/ 

-0.11567519939427878830185483678 . . . 



1 - 1/p 

1 + 1/p 



5.2 Numerical Computation 

Using the Dirichlet series given above for <P 4 ( 04 , 3 ), it is easy to obtain the 
formula 

N4(C4,X) = (5(A) + 5(A/16)+25(A/64)+45(A/2048)-A2(G2,A+2)_i)/ 2 , 
where N 2 (C 2 ,X) is given in Section 2 and 

S(X) = ImH|2++ E ■ 

p|n=>p=l (mod 4) gcd(m,2n) = l 

This formula can be improved in several technical ways, but basically it is the 
one that we have used. 
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5.3 Table 

In this table, we let P4^{Ci,X) = \c{C^^ + c'(C'4) be the predicted 

value and £4(04, X) = (IV4(C'4, X) — P4(C'4, X))/X^/^ rounded to 5 decimals. 



A 


474(4*4, A) 


74 ( 4 * 4 , A) 


Ea{Ca, X) 


10^ 


0 


0 


0 


1Q2 


0 


1 


-0.39811 


1Q3 


1 


3 


-0.50238 


10-^ 


10 


10 


0 


10® 


32 


33 


-0.10000 


10® 


113 


no 


0.18929 


10^ 


363 


361 


0.07962 


10® 


1168 


1167 


0.02512 


10® 


3732 


3744 


-0.19019 


lOi® 


11930 


11956 


-0.26000 


loll 


38045 


38060 


-0.09464 


1012 


120925 


120896 


0.11545 


1013 


383500 


383472 


0.07033 


1013 


1215198 


1215158 


0.06340 


lOi® 


3848219 


3848077 


0.14200 


lOi® 


12180240 


12180346 


-0.06688 


1017 


38542706 


38542753 


-0.01871 


lOi® 


121936924 


121936998 


-0.07400 


1019 


385715463 


385715227 


0.16078 



6 Degree 4 Fields with G ~ V 4 = C 2 X C 2 



6.1 Dirichlet Series and Asymptotic Formulas 

By studying discriminants of biquadratic quartic extensions, it can easily be 
shown that 




From the above formula, we can easily deduce a crude form of the asymptotic 
formula: 

^4(14, X) ~ c{Va) log^ A with 

c(F 4 ) = ^ n ^ 0-0027524302227554813966383118376 . . . 
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It is easy to refine this to the more precise result 

X) = ( 0 (^ 4 ) log" ^ + c'(y 4 ) logx + c"(y4))^^/" + o{x^) 
for any a > 1/3, with 






p>3 



(p- l)(p + 3) 






40(14) 



24cW)|^7.4-§log»2-4i: 



P>3 



{p- l)"(p+ 3)^ 



where 7 is Euler’s constant and 



log k log" n 



7 i = lim > , 

n^oo \ k 2n 

\k=l 



= -0.0728158454836767248605863758749 . . . 



Numerically, we have 

c'(E 4 ) = 0.05137957621042353770883347445 . . . 
c"(E 4 ) = -0.2148583422482281175118362061 . . . 

It is reasonable to conjecture that we can in fact take any a > 1/4 in the 
above asymptotic formula. 



6.2 Numerical Computation 



From the Dirichlet series given above for <^ 4 ( 14 , s), we obtain easily that 

m{V4,X) = ^ ^ /^(„)|^(„)|3-(")-lfV2(C'2,Vx)-i , 

n<VX 
n odd 



where 



fx{n) 



{ 16 if l<n<v^/16 
10 if v^/16 <n< Vx/8 
4 if VX/8 <n< Vx/4 
1 if VX/4 <n<VX. 



Together with the formula given for 44 (C 2 , 41) in Section 2, this is the formula 
that we have used. 
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6.3 Table 

In this table, we let P4(V4, X) = \_{c{V 4 ) \o^ X + c' {V 4 ) log X + c" be 
the predicted value and X) = X) — Pi{Vi, X))/X^/'^ rounded to 

5 decimals. 



A 


N4{V4,X) 


P4{V4,X) 


E4{V4,X) 


10^ 


0 


0 


0 


1Q2 


0 


1 


-0.31623 


1Q3 


8 


9 


-0.17783 


10^ 


47 


49 


-0.20000 


10® 


243 


234 


0.50611 


10® 


1014 


1020 


-0.18974 


10^ 


4207 


4201 


0.10670 


10® 


16679 


16655 


0.24000 


10® 


64316 


64255 


0.34303 


lOi® 


242710 


242751 


-0.12965 


loll 


901557 


901967 


-0.72909 


1012 


3306085 


3306219 


-0.13400 


1013 


11982067 


11982984 


-0.51567 


1013 


43017383 


43016720 


0.20966 


lOi® 


153156284 


153154732 


0.27599 


lOi® 


541382988 


541386997 


-0.40090 


1017 


1901705324 


1901714182 


-0.49812 


lOi® 


6642813777 


6642812780 


0.03153 


1019 


23087994312 


23087989990 


0.07686 



7 Degree 4 Fields with G ~ D 4 



7.1 Dirichlet Series and Asymptotic Formulas 

In [3] we prove that 

<p4{D4, s) = X! s) - I) - -<p4{V4, s) - -^4(^4, s) , 



where we sum over all quadratic discriminants D and the formulas for <^4(14,5) 
and <?4(C'4, s) can be found in Sections 6 and 5 respectively. The Dirichlet series 
^2,d{C2, s) is defined as follows. Set K = Q(Vd), let T 2 {K) be the number of 
nonreal places of K (i.e., 1 if D < 0 and 0 if D > 0), and denote by C/f(s) the 
Dedekind zeta function of K. Then 



^2,d(C'2, s) 



24s-2+r2(/C)^^(2s) 






c|2 




where y runs over all quadratic characters of the ray class group 01^2 (K) corre- 
sponding to the modulus c^. 
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From the above formula, we can easily deduce a crude form of the asymptotic 
formula: 

fV4(£>4, X) ~ c{Di) X with 



c{D4) 



7r2^2’'2(^)L)2^(2, (^)) 



n>l D\n 
DGZ 



D 



\n/\D\ 



Hn/\D\) 



where the sums on D are over all discriminants D of quadratic fields, T 2 {D) = 
?'2(Q(\/^)) as above, and finally L(s, (— )) is the L-series associated to the 
Legendre-Kronecker symbol (■^)- 

It is possible that the constant 0(1)4) can be expressed as a finite linear com- 
bination of Euler products with explicit coefficients, but we have not been able to 
find such an expression. Consequently, we must sum over all quadratic discrim- 
inants to obtain a numerical value, and use standard extrapolation techniques. 
In this way, we obtain numerically 



c(D 4) = 0.052326011... 



where the last digit may be wrong. 

In view of the way in which we have obtained the Dirichlet series <^4(1)4, s), 
we can conjecture that more precisely 

N4{D4, X) = c{D4) X-^ {c{V4) log" X + c'{V4) log X + c" {V4) + X^/^ 

+ 0(X“) 

for all a > 1/2. The numerical data suggest that the error term can be re- 
placed by (c'(D4)logX -I- c"{D 4 ))X^/'^ + for suitable constants c'(D4) 

and c"{D 4 ). 



7.2 Numerical Computation 

Using the methods explained in [6], it is not difficult to count the number of 
quadratic extensions of a given base field K, since these are of the form 
for suitable values of a. Hence ^4(1)4, X) is equal to the sum over all fundamen- 
tal discriminants D yf 1 of the number of quadratic extensions of Q{\/D) whose 
relative ideal discriminant has a norm less than or equal to XjD"^ . 



7.3 Tables 

In the first table, we let 



P4{D4,X) 



c(i)4)^-^(^c(U4)log"(X) + c'(U4)logX + c"(U4) + ^^)^'/" 
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be the predicted value and £ 4 ( 04 , X) = (7V4(_D4, X) — P 4 {D 4 , X))/X^/'^ rounded 
to 5 decimals. However, in view of the way in which we have obtained the asymp- 

3 1 

totic formula, we can also set N 4 {I, X) = N 4 {D 4 , X) + -N 4 {V 4 , X) + -N 4 {C 4 , X) 
(where I stands for imprimitive), and compare it with P 4 (I,X) = [0(1)4) X]. 
Hence, in the second table, we set £ 4 ( 1 , X) = (N 4 {I,X) — £ 4 ( 1 , X))/ X^/'^ 
rounded to 5 decimals. 



A 


N4(D4,X) 


P4(Di,X) 


Ei(D4,X) 


10 ^ 


0 


1 


-0.31623 


10 ^ 


0 


3 


-0.30000 


10 ® 


24 


38 


-0.44272 


10 ^ 


413 


443 


-0.30000 


10 ® 


4764 


4862 


-0.30990 


10 ® 


50496 


50734 


-0.23800 


10 " 


516399 


516766 


-0.11606 


10 ® 


5205848 


5207008 


-0.11600 


10 ® 


52225424 


52227698 


-0.07191 


10 ®° 


522889160 


522889735 


-0.00721 



A 


N4(I,X) 


P4(7,A) 


Ei{I,X) 


10 ® 


0 


1 


-0.31623 


10 ® 


0 


5 


-0.50000 


10 ® 


36.5 


52 


-0.49015 


10 ^ 


488.5 


523 


-0.34500 


10 ® 


5144.5 


5233 


-0.27986 


10 ° 


52073.5 


52326 


-0.25250 


10 " 


522891 


523260 


-0.11669 


10 ® 


5231450.5 


5232601 


-0.11505 


10 ® 


52323764 


52326011 


-0.07106 


10 ®° 


523259190 


523259964 


-0.00920 



In these tables, the behavior of the error term suggests that there is an 
additional main term, that our theoretical methods are unable to explain. As 
mentioned above, it is probably of the form (c'(D4)logA + c"(D4))A^/^ for 
suitable values of £{£ 4 ) and £'{£ 4 ). 



8 Degree 4 Fields with G ~ A 4 



8.1 Dirichlet Series and Asymptotic Formulas 



Using our usual Kummer-theoretic method, we have obtained an explicit ex- 
pression for the Dirichlet series <^4(44,5) which involves a sum over quadratic 
characters of ray class groups of cyclic cubic fields with modulus dividing 4, and 
which is too long to be given here. 

From this expression, we can easily deduce a crude form of the asymptotic 
formula: 

N 4 {A 4 , X) ~ 0(44) X^!"^ log X with 



0(44) = lim 

N^OO 



1 



31og2C(3) 



E 



K 3 

N<f{K3)<2N 



h{K3)R{K3)c2{K3)CriK3) 



P{KY 



with 



P{KY 



n 

p split in K3 



(1 + 3/p)(1-1/p)2 

l + l/p+l/p^ 



where A3 ranges over all cyclic cubic extensions of Q up to isomorphism (which 
can easily be described explicitly, for example by using the Dirichlet series for 
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C3 extensions given in Section 3), /(i^a), h{K^), R{K^) denote the conductor, 
class number and regulator of 



CriKs) 



n 

PlfiKs) 



1 

1 + l/p+ 



and C 2 {K^) = 11/8 if 2 is inert in it's, while = 23/20 if 2 is totally split 

in K^. 

As for the case G ~ D 4 , it is possible that the constant 0(^4) can be expressed 
as a finite linear combination of Euler products with explicit coefficients, but we 
have not been able to find such an expression. Consequently, we must sum over 
all cyclic cubic extensions of Q to obtain a numerical value. We obtain the very 
poor value 

c(A 4) = 0.017892 

It should not be too difficult to obtain an improvement of the asymptotic 
formula to 



N4{A4,X) = (c(A4)logA + c'(A4))Ai/"+0(A“) 

for a > 1/3. The tables seem to give something like d{A 4 ) = —0.12354. 



8.2 Numerical Computation 

We could have used directly the Dirichlet series mentioned above. However for 
simplicity and also because we lose at most a time factor of 10, we have preferred 
to generate A 4 extensions using Kummer theory of quadratic extensions over 
cyclic cubic fields and keep only those extensions whose discriminant is less than 
the required bound (see [4] for details). 



8.3 Table 

In this table, we let ^4(^4, X) = [0(^4) log A] be the predicted value and 
£14(^4, A) = (A4(A4, A) — P4(A4, A))/A^/^ rounded to 5 decimals. 



A 


N 4 {A 4 ,X) 


Pa{A 4 ,X) 


E 4 {A 4 ,X) 


10^ 


0 


0 


0 


10^ 


0 


0 


0 


10^ 


0 


0 


0 


10-^ 


4 


4 


0 


10^ 


27 


26 


0.02154 


10® 


121 


124 


-0.03000 


10^ 


514 


521 


-0.03249 


10® 


2010 


2060 


-0.10772 


10® 


7699 


7818 


-0.11900 


lOlo 


28759 


28844 


-0.03945 


loll 


104766 


104241 


0.11311 




282 



Henri Cohen, Francisco Diaz y Diaz, and Michel Olivier 



9 Degree 4 Fields with G ~ S 4 

9.1 Dirichlet Series and Asymptotic Formulas 

By using similar methods to the A 4 case but this time with Kummer theory over 
noncyclic cubic fields, we have computed explicitly the Dirichlet series ^ 4 ( 34 , s), 
which is quite similar in form to <^4(44,5). We can easily deduce from this 
Dirichlet series the crude asymptotic formula 

N4(S4,X) ~ c(54) A 

but the expression for the constant c(5'4) is too complicated to be given here and 
is not easily computed numerically since we must sum over all noncyclic cubic 
extensions of Q. Hence, contrary to the other Galois groups, it is for the moment 
difficult to give similar tables to the ones given above. 

As for the case G ~ D 4 , it is possible that the constant c(S'4) can be expressed 
as a finite linear combination of Euler products with explicit coefficients, but 
we have not been able to find such an expression. Wright and Yukie (private 
communication) assert that they have such an expression, not yet completely 
proved, but the error term is so large that the amount of data that we have 
is insufficient to check whether their expression is plausible. A combination of 
their work with experimental data suggests that we could have (but this is to 
be taken with a huge grain of salt) 

^4(^4, X) = 0.6382 A - 0.764 + 0(A“) 

for some a < 0.97, perhaps any a > 1/2. 

9.2 Numerical Computation 

As for A 4 extensions, we use Kummer theory of quadratic extensions, this time 
over noncyclic cubic fields and we keep only those extensions whose discriminant 
is less than the required bound. See [4] for details. 

9.3 Table 

As mentioned above, we give the exact values of N 4 {S 4 ,X) together with the 
value P4(54,A) = [0.6382 A - 0.764 AO-9^] and £ 4 ( 84 , X) = (A4(54,A) - 
^4(54, A))/A^/^, but it should be once again emphasized that contrary to 
the other Galois groups, these cannot be considered as predictions but just as 
guesses. 



A 


N4{S4,X) 


7^4(54, A) 


£ 4 ( 84 , X) 


10^ 


0 


-1 


0.31623 


10^ 


0 


-3 


0.30000 


103 


18 


17 


0.03162 


10^ 


570 


586 


-0.16000 


10® 


9739 


9733 


0.01897 


10® 


133322 


133430 


-0.10800 
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Abstract. Let L be a number field and o be an ideal of some order of 
L. Given an algebraic number a mod o and some bounds we show how 
to effectively reconstruct a number b if it exists such that b is smaller 
then the given bound and b = a mod o. 

The first application is an algorithm for the computation of n-th roots of 
algebraic numbers. Secondly, we get an algorithm to factor polynomials 
over number fields which generalizes the Hensel-factoring method. Our 
method uses only integral LLL-reductions in contrast to the real LLL- 
reductions suggested by [6,8]. 



1 Introduction 

One of the most basic methods in algorithmic number theory is to work “mod- 
ular”, i.e. instead of looking for a solution a G Zp the ring of integers (or any 
other order) of the number field F, one examines the problem modulo a suitable 
ideal a. More precisely, one looks at the canonical epimorphism (/) : i? — > R/a 
and considers the “easier” problem in R/a. 

Suppose we have a solution f3 mod a and want to lift it to a solution a ( “re- 
construct” a from (3). In order to do this we need some additional information. 
In this paper we focus on lattice based techniques, the additional information 
will be a bound on the “size” of a. This enables us to find a as a smallest element 
in the coset mod a - provided a is “large” enough. 

This idea has already been used in the literature ([3,6,8]). The difference to 
our approach however is the use of a different lattice (a different size function). 
Our choice of the lattice allows us to work with integers only. 

This new reconstruction has already been used successfully for factoring poly- 
nomials over number fields [6] and computation of roots in class field computa- 
tions. Other applications are the computation of embeddings of number fields 
and irreducibility testing for polynomials. 

We don’t use the canonical Minkowski-map to embed the ring of integers into 
a real vector space. Our embedding depends on the chosen Z-basis of the ring of 
integers. Although this seems to be a theoretical disadvantage it yields a good 
computational behavior. We have implemented our method in the computer 
algebra system KASH [4] and provide some numerical examples which point out 
the advantages of our method at the end of this article. 



W. Bosnia (Ed.): ANTS-IV, LNCS 1838, pp. 285-296, 2000. 
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2 Successive Minima 

Let us fix some notation first. Through the remainder of this article K will be a 
number field of degree n (over Q) and R will be a fixed order of K. All lattices 
(which will be denoted by A in the following) are considered to be subsets of M” 
and are equipped with the Euclidean norm ||.|| as length. 

Further we choose a Z-isomorphism (group-isomorphism) 

S-.R-^A. (1) 

By Q (or Qzi if we want to emphasize the lattice involved) we denote the 
quadratic form, occasionally viewed as a mapping Qzi • R which 

is achieved by applying the scalar product (of M”) to the image of the Z- 
isomorphism S. The first successive minimum Ai(Z\) is the square of the length 
of the shortest non-zero element in the lattice A. 

We fix a basis wi , . . . , for i? as a Z-module, this also yields a Q-basis for 
the number field K. For an integral ideal a of R, A(a) denotes the sublattice of 
A corresponding to a as a submodule of R. 

Let us fix an element P G R and some real number c > 0. Our task is now to 
decide if there exists any a G R such that a — P G a and Q{a) < c. 

We begin with the following trivial lemma: 

Lemma 1. Suppose the first successive minimum Ai(Z\(a)) is greater than 4c. 
Then there exists at most one a G R such that Q{a) < c and P — a G a. 

Proof. Suppose we have «i and 02 with the desired properties. Then a\ — a 2 = 
oi — /3-I-/3 — a 2 Ga and a/ Q{ai — 02 ) < \ZQ(o:i) + \/Q{a^ < ‘2.^/c, implying 
Q{a\ — 02 ) < 4c < Ai(Z\(a)). Therefore we get 0 = — « 2 - □ 



Remark 1. Under the assumptions of lemma 1, it is possible to compute a as 
the lattice point closest to P by well known enumeration procedures [7]. 

Since the enumeration part has potentially exponential running time and is 
therefore potentially slow, we show how to avoid enumeration. We need another 
lemma from [3] : 

Lemma 2. Suppose b\, . . . , is a LLL-reduced basis of the lattice A. Then we 
obtain min^gRnj^H^i Q(X) > Ai(Z\)2“"("“B/4^ 

The LLL-reduction [5] can be found in most text books on computational 
algebraic number theory, e.g. [7] which also contains an algorithm for computing 
a LLL-reduced lattice basis. The idea of the LLL-reduction is to construct a 
basis which is as orthogonal as possible. Let us remark that a basis of the order 
R or the integral ideal a induces a basis of the corresponding lattice and vice 
versa. The matrix transforming the lattice basis to a LLL-reduced lattice basis 
can also be applied to get the corresponding order basis resp. ideal basis. 

Having the above lemma in mind, the following is straightforward: 
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Lemma 3 . Suppose c < is a LLL-basis for 

Z\(a) and let f3 = X^r=i ^ V there exists a € P + a with 
Q{a) < c then a = ~ 

(We write [r] = 2 to denote the unique 2 G [r — 1 / 2 , r + l/ 2 [ n Z.) 

3 Reconstruction Using the Method of Pohst— Roblot 

In order to use the ideas of the preceding section we have to specify the lattice 
and to show how to choose the ideal which amounts to give an estimate for 
Ai(a). Both Pohst [6] and Roblot [8] use the Minkowski-map to map R into M”: 
Let F = Q(7) be given via a primitive element 7. The conjugates 7^*^ of 7 are 
ordered in the usual way: 7*-^\ G R and = ^(ri+r2+i)^ 

^(ri+r2) _ ^(ri+r2+r2) g C \ M. Then we define 

( 7*^®) 1 < i < ri 

ri + 1 < z < ri + T2 (2) 

■\/2iA(7(*“’’=^)) ri+r2 + l<z<n 

and Z\r := Sm(R). We get Qak(q:) = Qm(q:) := T2(a) := 

For the reconstruction process Pohst applies lemma 1 whereas Roblot makes 
use of lemma 3 . In addition, both use the following: 

Lemma 4 . For every non zero 7 G a zee have 22(7) > rz-/V(a)^/”. 

Proof. We have N{'y) > N{a). Now, by the inequality between geometric and 
arithmetic means we get: 

n 

(IV( 7 )f/" = -7^2(7) • ( 3 ) 

i=l 

□ 

Using ideals of the form for a fixed prime ideal p it is now easy to compute 
the exponent necessary to ensure Ai(p^) > d for any d > 0. 

As demonstrated in [6,8] this can be used to get an efficient factoring algo- 
rithm for polynomials over number fields. 

Since both Pohst [6] and Roblot [8] work with real lattices, their algorithms 
suffer from the usual problems with real arithmetic in computer algebra systems. 



4 The New Approach 

To overcome the precision problems, we choose a different lattice. Since i? is a 
free Z-module of rank n, it is natural to consider Az := Z” with the usual scalar 
product. Qz{oi) now measures the size of the coefficients of a if represented as a 
linear combination of wi, . . . , 
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The two isomorphisms '■ R ^ and 6 k : R ^ M” induce a third 
isomorphism %p : Ai, Z\r : ip := (5r o <5^ This isomorphism is also obtained 
using the “real” basis of R-. ip{x) = M ■ x, where M = ((5r(wi), . . . , <5r(w„)) for 
a fixed basis wi , iVn of R. 

The quadratic form on Z\r resp. Z\z will be denoted by Qr(.) resp. Qz{-)- 

Since ip and ip~^ are continuous (when considered as maps form M” ^ M”) 
we get constants ci, C 2 G M such that QR(a;) = ||(5R(a;)|p = \\ip o <5z(a;)|p < 
ci||(5z(a;)|p = ciQz(a;) for all a; G i? and Qz{y) = ||<5z(y)|P = \\^~^ o <^R(a;)|P < 
C2||<5R(a;)|p = C2Qr(3;) for all y G -R. It is well known from (numerical) analysis, 
(see e.g. [9]) that the smallest possible Ci are obtained as the largest (ci) resp. 
smallest (c^^) eigenvalue of M* • M. We note that the eigenvalues of M* • M are 
real and positive. 

Now we have to consider the sublattice Z\z(a) corresponding to the ideal a as 
a submodule of the ring R. The sublattice Az(a) is generated by the columns of 
the “transformation” matrix B G Z”^” i.e. a basis of the Z-module a is obtained 
via (oi, . . . , ttn) = (oJi , . . . , uJn) ' B* , and the columns Bi., . . . , R„. of B form a 
basis of the sublattice Az{a). 

Provided with this we are able to apply lemma 1 and 3, i.e. given an upper 
bound for the size of the coefficients of a and an ideal with first successive 
minimum satisfying the condition of lemma 1, then we can reconstruct a via 
enumeration in the lattice Az{a). If the condition of lemma 3 is satisfied, we can 
reconstruct a in the LLL-reduced basis of Az(a) via rounding of the coefficients 

(L-D- 

So we need a way to estimate the first successive minimum of Z\z(a). 
Lemma 5. The following lower hound for the first successive minimum holds: 

Xi{Az{a)) > (4) 

Cl 



Proof. 



Ai(Z\z(a)) = min Qz{z) > min — Qr(2) 

2Ga\{0} 2Ga\{0} Cl 

= -Ai(ZiR(a)) > -nfV(a)"/". 

Cl Cl 

The last statement is a consequence of lemma 4. □ 

If we are given an upper bound for the T 2 -value of a, we can easily calculate 
an upper bound for the size of the coefficients of a. 

Lemma 6. Provided that T 2 ( 7 ) < c for j G R we have Qzfci) < C 2 • c. 

Proof. Immediate. □ 

Collecting all this we get the following theorem: 
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Theorem 1. Let a he an ideal such that 



fV(a)>^ ^ ) 



(5) 



and b\, . . . , he a LLL-reduced basis of the lattice Az{a). Let /3 G R he arbitrary. 
Then (3 = X^r=i with some qi G Q. Furthermore, we assume that there 

is an a G P + a such that QR(a) < c. 

Then we get a = /3 — 7, where 7 := Y17=i 

Proof. Assuming that 



7V(a) > ^ j 



( 6 ) 



lemma 5 yields 



Ai(Az(a)) > 



C^^2^(u-1)/4+2 



n 



(7) 



Now we apply lemma 3 to obtain the unique element 7 := Y17=i if if 

exists and conclude that Qz{P ~ l) < cci. 

Lemma 6 shows that a = /3 — 7. □ 



This new approach has the advantage of purely integral computations while 
dealing with the lattice Ai, i.e. LLL-reduction and enumeration of the closest 
lattice points. For the sake of this we get worse bounds (larger exponents of 
the prime ideal p) than Pohst [6] and Roblot [8]. The computation of the ideal 
a = p^ can be done very efficiently using the ideas of [6], so that the worse 
bounds do not yield a disadvantage of our method. This is also demonstrated 
by the numerical examples. 



5 Denominators 

In order to extend our method to the reconstruction of non-integral numbers 
we have at least two choices: to convert the problem into an integral one (by 
multiplying everything with a suitable integer, choosing a different polynomial, 
etc.) and using a bound on the denominator, to reconstruct it. This method is 
therefore useful for reconstructing elements not contained in the equation order. 

We consider the following situation: We want to find ajd with a G R and 
d gN. We know bounds for a and d: Qz(a:) < B and df < B, d^ a and assume 
that we have already computed P G R such that dP — a G a. 

In order to compute a and d we make again use of the LLL-reduction, we 
extend the lattice Az using the following map: 

■ (8) 

Let A'(a) := {i{Az{a), P'), by P' we denote the vector (l,/3i,... ,/3„)* where 
/3 = E”= PiUJi and , . . . Wn is the Z-basis of R. In order to simplify the notation 

we use Q' instead of QA'{a) quadratic form on the lattice A'(a). 
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Lemma 7. If the ideal a has first successive minimum Ai(Z\^(a)) > then 
there is at most one to G subject to Q'{iS) < Bi. 

Proof. Suppose oji,oj 2 G with Q'(uJi) < Bi. The elements oji are of the 

form uji = fiP' +t.{Sz{ai)) with /j G Z, Oi G a, z = 1, 2. An immediate consequence 
of the bounds Q'(uJi) < B\ is fi < 

There exists an element a G 2iz(a) such that z(d) = / 2 W 1 +/ 1 W 2 G z(Z\z(a)) C 

Qz(d) = A/Q^^ 7 (d)y = Q'{f 2 ^i + /1W2) < /2\/Q (wi) + /i a / Q (1-O2) 

< (/i + f 2 )\fBi < 2Bi We conclude Qz(a) < 4Bf < Ai(Az(a)) such that d = 0 
and oji = 0 J 2 - □ 



Theorem 2. Suppose that the ideal a has first successive minima Ai(Z\^(a)) > 
and there exists a and d such that d/3 — a G a, then the shortest vector of 
the lattice 2i'(a) is to = d(3' + z(<5z(<a)). 

Proof. We know that a = d(3 mod a so there exists an element d G a with 
the property a = d(3 + d. The element u> := df3' + z(<5z(d)) = dei + z(<5z(<a)) 
where e\ is the first canonical basis element of It is easy to see that 

Q'{uj) = df + Qz(a) < 2B. Using B\ = 2B in lemma 7 we obtain that u must 
be the shortest vector in Z\'(a). □ 



Remark 2. If the bounds of theorem 2 are suitably enlarged, we can use the first 
basis vector of a LLL-reduced basis for Z\'(a) to get a solution. 

In contrast to the preceding version (theorem I ) we need a LLL-reduction for 
each number to be reconstructed. In theorem I we only need one LLL-reduction 
for the ideal basis and can use rounding to reconstruct any number of inte- 
gers. We want to remark that this method (theorem 2) also uses purely integral 
computations only, i.e. the lattice N C Z”+^, because all the fii G Z. 

6 Numerical Improvements 

The value ^Jc\C 2 occurring in theorem 1 is known in numerical analysis as the 
condition cond(M) of M. This is used as a measure for the numerical difficulty 
of the matrix [9] . 

Since cond(M)” is a factor of our bounds, we should try to reduce it. Ob- 
viously, cond(M) = 1 if M is an orthogonal matrix. Therefore one should start 
with a LLL-reduced basis for Ak{R) in order to “make M as orthogonal as 
possible”. Since cond(M) is independent of the application (i.e. part of every 
estimate used) we are permitted to spend some time on the computation (of a 
good estimate) of cond(M). 

We want to point out three different methods to compute (estimates for) the 
condition cond(M), the first is related to algebraic number theory, the second 
and the third are suggested by numerical analysis [9,10]. 
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6.1 Characteristic Polynomial 

Every computer algebra system contains a function to compute the characteristic 
polynomial and another function which computes the real zeroes of a polynomial. 
So we can easily make use of this: 

1. Compute the characteristic polynomial Xm*-m{x) of the matrix M*M, 

2. compute the real zeros of Xm*m(x) '. A„ > . . . > Ai, 

3. the condition will be cond(M) = a/ A„/Ai. 

6.2 Von-Mises-Iteration 

If the matrix M^M is equivalent to a diagonal matrix and the largest eigenvalue 
Amax occurs with frequency one, we can construct a sequence at := ^ , where 

ti := A^'to converging to the largest eigenvalue, provided that the input vector 
to is not orthogonal to the eigenspace of Amax- The von-Mises-Iteration is also 
known as the “power-method” [10, p.47]. 

We have to apply the von-Mises-Iteration also to the inverse of the matrix 
M*M, to obtain the smallest eigenvalue Amin of M^M. This method suffers from 
the slow convergence in the case A„_i (the second largest eigenvalue) is very close 
to Amax- 

6.3 Matrix-Norm 

This method is very simple and known as the theorem of Hirsch [10, p.81]: 

Theorem 3. Assuming that the matrix norm ||.|| is admissible for the (Eu- 
clidean) vector norm (i.e. ||Ma ;||2 < \\M\\ l|3:||2j; then the inequality Amax < 
||M*M|| holds. 

So we can use for example the Schur-norm 

Pllschur := ( E (9) 

\i,k=l j 

which is admissible for the Euclidean vector norm [9, p.l67]. This yields the 
following upper bound for the condition of M : 

COnd(M) < ||M‘M||sehur||(M‘M)-l||sehur - (10) 

7 Applications 

In this section we investigate different applications of the new reconstruction 
method, i.e. factorization of polynomials over number fields, irreducibility-testing 
of polynomials over number fields and computation of r-th roots of algebraic 
elements. We demonstrate the algorithms and point out some improvements. 

In order to use the developed theory, we apply an algorithm of the following 
shape: 
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1. task: compute a € R with a certain property 

2. compute a bound for the result, usually T 2 {a) < c 

3. pick a suitable prime ideal p of i? 

4. compute k such that a p^ matches either theorem 1 or theorem 2. 

5. compute an approximation /3 of a mod a, this usually involves a Hensel- or 
Newton-lifting 

6. use theorem 1 or 2 to either find a or show that it does not exist 

Depending on the actual problem the details vary. Suppose we know that a 
exists. Then we can simply try to compute a mod p^ for increasing k until we 
are successful. Since the bounds are usually much too large, this gives quite a 
speedup. 

One often tries different prime ideals, e.g. to get a first degree prime ideal 
(to simplify the lifting) or to get special factoring shapes. We will discuss this 
for each problem individually. 

7.1 Factorization and Irreducibility- Testing 

Let / € be a square-free monic polynomial and R be the maximal order of 
some number field F. We want a complete factorization of /. Since / is monic, 
the factorizations over R and over F coincide. From [6] or [1] we get estimates 
for the coefficients of a factor g of / as follows: 

Theorem 4. Let f = x'^ + ^ ® polynomial and g = + 

a proper factor. Then 

( 11 ) 

for all i where := Q 

Furthermore, it is well known that if p is no divisor of the discriminant of / 
then / mod p is square free, too. Therefore we can obtain a factorization of / 
modulo p^ using linear or quadratic Hensel-lifting. 

The algorithm is straightforward: Compute all possible factors mod p^ for 
a suitable k, apply reconstruction to each coefficient and test if one gets true 
factor in R[x\ this way. To make this efficient, we use ideas from [2] for early 
factor detection and some heuristics for the enumeration of all factors. 

The Irreducibility-Testing is just a special case of the general factoring al- 
gorithm. But the computation can be speed up, if one stops after the first true 
factor is found. 

7.2 Roots 

This too can be viewed as a factorization problem. However, due to special shape 
of the polynomial we get sharp bounds for the factors. Furthermore, since we 
are only looking for linear factors, we need no recombination step. The lifting 
necessary for the root computation can be done using quadratic Newton lifting. 
When working without bounds it is advisable to use p*-^ rather than p^. 
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8 Examples and Discussion 

The numerical examples in this section demonstrate the advantages of the new 
method. For computations we use the computer algebra system KASH [ 4 ] on a 
Intel Pentium III ( 600 MHz, 5 I 2 MB RAM) running under Linux 2 . 2 .I 3 -SMP. 

In the case of totally real fields one can get rid of the real arithmetic by using 
LLL-reduction on the gram matrix, which has integral entries in this case, instead 
of using LLL-reduction on the real-basis of the lattice. The implementation of 
Pohst’s algorithm [6] benefits from this fact. This will be demonstrated by the 
following two examples which are taken from [6] . 

8.1 Factorization 1 

Let F := Q(7) generated by a root 7 of the polynomial f = — 28 t^° — 

40 f 9 -b 180 t® -b 426 t^ -b 89 t® - 444^^ - 390 t^ - + 27 -b 1 It -b 1 . This polynomial 

splits completely over the number field F . 

It takes 170 ms to factor / over F using the algorithm of Pohst [6]. The 
creation of the real lattice (including the LLL-reduction) takes 50 ms, whereas 
our new method takes 210 ms. 

For the computation both algorithms use the following prime ideal and 
bounds: p = 654497 ? -b (2545 -b 7)??, where R is the ring of integers in F, and a 
T2-bound of 1344 for the coefficients of linear factors yielding an exponent of 4 
for the prime ideal p. 

Using a LLL-reduced basis for R and the complex zeroes for the character- 
istic polynomial, we obtain cond(M) < 4 . Applying the Schur-norm, we get 
cond(M) < 27 . This increases the bound for the exponent to 12 resp. 13 . 



8.2 Irreducibility- Testing 

The polynomial 5 := ?i4+^(_2l39+25107-bl57372-141973-b38874-b76075- 
33476 - 20437 ^- 129278 -b 49579 -bl 77 i°- 127 ii)fi 3 +^( 222 - 15177 - 180072 -b 
143673 -b 9367 ^ - 28057 ® + 297 ® -b 3757 ^ -b 10067 ® + 977 ® - 657!° -b + 

(4798 - 457 - 972 -b 70273 - 24973 - 3 1 257® - 14576 - 1 37^ -b 257278 -b 75476 -b 
3I7I6 -b 167ii)fii -b 1^(3842 -b 33337 + 261476 -b 195773 - 113573 -b IO67® -b 
I6I476 -b 3387^ -b 229578 -b 54276 - 7I7I6 -b Il7ii)fi6 -b y ^(-4552 - 63967 - 
379576 - 127378 - 638673 - 20497® - 37776 - 12497^ - 234778 - I3IO76 - 867I6 - 
13711)^6 -b y ^(3996 -b 37747 -b 333576 -b 442178 -b 275O73 -b 23257 ® -b 14257 ® - 
16437 ^ -b 85378 -b 57O76 -b 627I6 - 371 1 )f8 -b (2266 -b 44477 - 90476 -b 460778 - 
91973 -b 31527 ® -b 107276 - 3047 ^ -b I25I78 -b 67876 -b 57I6 -b + i ^(1324 -b 
22887 + 3O6376 - 150978 - 55273 - 43697 ® + 19976 -b 4957^ -b 7857 ® -b 8I676 -b 
227I6 -b 177 ii)f 6 -b i ( 3347 -b 7557 -b 42476 -b 103278 -b 343I73 -b 4197 ® -b 10257 ® -b 
1 3007^ -b 647 ® -b 72576 - 1 37I6 -b IO71 1 ) t® -b ^ (35 - 187 - 66276 - 68278 - 8O273 - 
2297® - 22376 - 2907^ -b 17778 - 776 - 237I0 - 71 1 )?3 -b (326 - 60737 -b 2OI876 - 

8OO378 - 542I73 - 29727® - 155676 - 5687^ - 38I78 - 132876 - I577I6 + 

Y ^(730 - 537I7 -b 467O76 - 23978 - 224573 -b 6567 ® + 4407 ® - 5087^ - 12517 ® - 
55276 - 897I6 - 7ii)f6 -b y ^(2930 -b 31117-b 472O76 -b 85978 - 386773 - 6907 ® + 
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120276 + 11547^ + 44578 + 7276 + II7I0 + 197ii)i + j^(-3308 - 37 + 25747^ - 
58OI78 + 33974 + I9O876 - 93676 - 16587^ - 135578 - 60476 + 4571° - 5744) 
of [6] is irreducible over F. It splits modulo p in 3 linear factors and one factor 
of degree 11. This leads to consider possible factors of degree one (T2-bound 
of 3051420855768 and exponent 15), two (T2-bound of 131313006031604 and 
exponent 17) and three (T2-bound of 525251952028508 and exponent 19). 

Depending on the estimate for cond(M) this induces exponent bounds of 23 
(24) for linear factors, 25 (27) for degree two and 26 (27) for degree three. 

It takes 310 ms to factor / over F using the algorithm of Pohst [6]. The 
creation of the real lattice (including the LLL-reduction) takes 140 ms, whereas 
our new method takes 440 ms. 

8.3 Factorization 2 

In the average case (not totally real), the new method takes advantage of the 
integer arithmetic and is much faster than the algorithm of Pohst [6] . We also 
compare our algorithm to a Trager [11] base method which is implemented in 
KASH [4]. 

Next we consider the field generated by a root p of — 10a;8 — Sx"^ — 

15a; + 10. This field has signature (3, 1) and discriminant —2282655415. As poly- 
nomials we choose “random” polynomials, i.e. we generate polynomials where 
the coefficients (of the coefficients) are bounded by B. We start by multiplying m 
polynomials of degree n and try to recover them using both our method, Pohst’s 
method and the Trager [11] based method. The numbers ti, t 2 and ts in table 
1 denote the time for our new method, Pohst’s method and the Trager based 
method in seconds. All are average times for several different polynomials. 



Table 1. Experimental results 



n 


m 


B 


ti 


t2 


U 


5 


3 


20 


0.5 


0.8 


0.71 


7 


3 


20 


0.6 


2.0 


1.4 


10 


3 


20 


1.2 


2.7 


2.8 


15 


3 


20 


1.4 


10.6 


6.47 


5 


5 


20 


1.2 


3.3 


2.4 


10 


5 


20 


3.0 


28.2 


11.6 


5 


3 


210 


0.4 


1.9 


1.0 



Finally we consider the field generated by a root p of / = x^^ — 16x^ + 
14a;8 + 2x'^ — 13a;6 -|- 18a;6 — 8a;4 — lla;^ -|- 6x"^ -I- a; — 5. This field has signature 
(2, 4), the equation order is already maximal. In table 2 we omit times for Pohst’s 
method. Because of the size of the involved numbers, the real precision necessary 
to recover (small) linear factors is enormous. Even to generate AR(a) takes more 
than 30 sec. In this case, we used a := (10737417417? -|- (577625038 -I- p)R)'^^. 
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Table 2. Experimental results 



n 


B 


ti 


t2 


5 


20 


1.3 


7.2 


10 


20 


3.0 


26.0 


15 


20 


3.5 


60.1 


20 


20 


4.2 


110.1 


25 


20 


5.6 


184.3 


5 


227 


3.6 


31.2 


10 


227 


4.8 


113.9 


15 


227 


8.4 


262.9 


5 


240 


6.4 


54.4 



8.4 Roots 

We consider the field F := Q(\/25601, Cr) which arises during the computation 
of the Hilbert class field of Q(-\/25601). One step in the algorithm requires us 
to take a certain 7th root of a smooth non-integral number 7 (smooth meaning 
the number has only small prime ideal divisors) . In this particular example, the 
number field has degree 12 over Q, the coefficients of 7 have approximately 420 
decimal digits and the denominator has 1030 digits (therefore we omit the exact 
value). Choosing a prime ideal p over 11 (of degree 3) we compute a mod p^ 
for fc = 2, 4, . . . , 1024 and using the results of section 5, it takes 12 seconds to 
compute the root. 

Clearing the denominator (i.e. computing the root of 7 x den( 7 )^ requires us 
to lift up to This process take 400 seconds. 

Clearing only the part of the denominator that is no 7th power (the denom- 
inator is 2^^® • 7®® • 41®®®, i.e. multiplying with 2®®® • 7^® • 41®®®) leaves us with a 
denominator of 7^. Now, using section 5 requires a precision of p^®® and a total 
runtime of 2.5 seconds. 
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Abstract. We describe a “dissected” sieving algorithm which enumer- 
ates primes in the interval [xi,* 2 ], using 0{xy^) bits of memory and 
using 0(®2 — a;i -I- arithmetic operations on numbers of 0(ln®2) 

bits. This algorithm is based on a recent algorithm of Atkin and Bern- 
stein [1], modified using ideas developed by Vorono'i for analyzing the 
Dirichlet divisor problem [20]. We give timing results which show our 
algorithm has roughly the expected running time. 



1 Introduction 

The sieve of Eratosthenes, or one of its many variants developed over the last 
few decades, remains the method of choice for locating primes in an interval 
[xi, X‘ 2 \^ provided the interval is sufficiently long. However, these methods appear 
to suffer from an overhead of operations as X 2 —> oo, making sieving 

inefficient for intervals much shorter than x^ . This puts a lower bound on the 
amount of memory required to sieve efficiently. More precisely, let us say that 
the interval [xi, x-^ has length l-|-a ;2 — a^i- Then, given B > 1 bits of memory and 
given a;i < a ;2 < a^i + B, we can sieve to find the primes p, x\ < p < X 2 , using 
0{{x2 — xi)xy^'^ +xy^~^°^^y operations. An interval of arbitrary length, perhaps 
longer than B, may be divided into several segments of length B, and possibly 
one more shorter segment. Summing the operation counts to sieve each segment 
gives a total of 0{xy^\x2 — a;i)(l + x^^ jB') + operations to sieve 

the entire interval, and we see that this is inefficient if B is much smaller than 
X 2 ■ A survey of sieving algorithms and their memory requirements is given in 
a recent paper by Sorenson [17]. 

When sieving the entire interval from 2 through x, memory requirements are 
unlikely to impede the calculation because the time to reach an x for which siev- 
ing becomes inefficient, say x « 10^*^, would be very great. For example. Nicely 
has been conducting a careful study of the distribution of prime gaps, twin 
primes, etc., and has taken roughly 7 years to sieve through 10^® [13,14]. How- 
ever, for other applications memory requirements may be a serious limitation. 
For example, the Lagarias-Odlyzko “analytic” algorithm for computing 7r(a;), 
requires the enumeration of primes in an interval about x, of length [11]. 

Although the Lagarias-Odlyzko algorithm is asymptotically the fastest known 
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method for computing 7t(x), it is expected that x must be quite large, perhaps 
as large as 10^^ or more, before this method becomes faster than other methods 
such as the extended Meissel-Lehmer method [10,2]. 

The primality of a given n can be determined using O(n^) arithmetic op- 
erations and O(n^) bits of memory, e.g., by using the APRCL or ECPP algo- 
rithms [15]. However, the cost per n appears to be significantly greater than 
the cost for sieving, provided we have enough memory and a sufficiently long 
interval to sieve. We describe an algorithm which sieves efficiently with signifi- 
cantly smaller memory requirements. This algorithm uses ideas from a recently 
developed algorithm of Atkin and Bernstein, which are then modified to give a 
“dissected sieve” that enumerates primes in the interval [xi,X 2 \ using 0 {x 2 ) 

1 /3 

bits and 0 {x 2 — xi + x^ ) arithmetic operations on numbers of 0 (lna; 2 ) bits. 

2 The Atkin-Bernstein Algorithm 

Atkin and Bernstein base their algorithm on classical theorems which relate pri- 
mality to properties of binary quadratic forms [1]. Theorem 1 below paraphrases 
their formulation. It uses a different but equivalent condition for Case (a), and 
is stated so that there is no overlap between the congruence classes considered. 

Theorem 1. Let n he a positive integer, and 

(a) if n= 1 (mod 4) let TZ = {(ui, U 2 ) : ui > U 2 > 0}, Q(ui, U 2 ) = Ui + 

(h) if n = 7 (mod 12) let TZ = {(ui, U 2 ) ■ ui > 0,U2 > 0}, Q{ui, U 2 ) = + 

(c) if n= W (mod 12) let TZ = {(ui, U 2 ) '■ u\> U 2 > 0}, Q{u\, U 2 ) = 

Let P{n) = #{(ui, U 2 ) € n 7^ : Q(ui, U 2 ) = n} . Then n is prime if and only 
if n is squarefree and P{n) is odd. 

For any n not divisible by 2 or 3, Algorithm 1 gives a method for determining the 
primality of a single n in 0{y/n) operations. The method can be made much more 
efficient when determining the primality of all n, a;i < n < a; 2 , provided X 2 ~ x\ 
and the available memory are large enough. (Just as the 0{y/n) method of trial 
division for a single n leads to the much more efficient sieve of Eratosthenes.) 

In the following discussion, pbuf is a “bit vector” data structure of B bits, 
indexed by n as pbuf[n] € {0,1}, and having two components pbuf.xl and 
pbuf . x2 giving the range of valid indices. We use the convention pbuf [n] = 1 if 
and only if n is prime. Given 3<a;i < n < X 2 < xi + B, Algorithm 1 below 
uses Theorem 1 to compute pbuf[n] = P{n) mod 2. (Algorithms 1 and 2 are 
meant to serve as subroutines for sieving segments of length < B when sieving 
an interval of arbitrary length.) 

After initializing pbuf[n] to zero, x\ < n < X 2 , Algorithm 1 enumerates 
lattice points (^ 1 ,^ 2 ) € TZ that are bounded between (or lie on) the conic sections 
Q{ui,U 2 ) = x\ and Q{u\,U 2 ) = X 2 , where TZ and the matching Q(ui,U 2 ) range 
through the three cases of Theorem 1. For each such point, the corresponding n 
is complemented when n is in the congruence class appropriate for the quadratic 
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form. Having computed pbuf [n] = P{n) mod 2, the algorithm sieves out numbers 
with square factors in a final pass. 

For a given quadratic form and associated congruence class, the lattice points 
within the swath x\ < Q{u\, U 2 ) < X 2 , (ui, U 2 ) G P-, are enumerated by varying 
ui, and then for fixed ui incrementing U 2 along a vertical scanline, choosing the 
starting and ending values of U 2 to avoid points outside the swath. Enumerated 
points are illustrated in Figure 1 and correspond to the dark points within a 
swath. (We show all lattice points within a swath, not just those satisfying the 
corresponding congruence condition.) 

Our algorithms are presented in a mixture of mathematical and C-like nota- 
tions. Assignment is denoted by , with a; -I— I- as shorthand for x *— x + 1. As- 
signments may be treated as expressions with values, as in (n ^ uf + uf) < X 2 , 
where the value assigned to n is then compared with X 2 - We write && for 
the conditional conjunction of boolean expressions, so that in the expression 
exprl && expr2, the second subexpression is evaluated only if exprl is TRUE, 
and the boolean value of the expression is the logical “and” of the two subex- 
pressions. Comments begin with “/ /” while important conditions or truths are 
emphasized with assert statements. 

Algorithm 1 Given a preallocated bit vector pbuf, indexed by n in the range 
3 < pbuf .xl < n < pbuf .x2, this algorithm sets pbuf[n] such that upon com- 
pletion we have pbuf[n] = 1 if and only if n is prime. 

1 SieveSegment ( pbuf ) { 

2 a;i^pbuf.xl; a; 2 ^pbuf.x 2 ; assert 3 < < a; 2 ; 

3 for ( n ^ a;i ; n < a; 2 ; n-|--l-) pbuf[n]^0; 

4 // Case (a) n = l (mod 4) , handles n mod 12 G {1, 5, 9} . 

5 for (ui ^ [(a;i/2)i/2] ; u? < 3 ^ 2 ; ui++) 

6 for ( U 2 <— |"max(0, xi — ; U 2 < ui && {n ^ < X 2 ; U 2 ++) 

7 if (n mod 4=1) pbuf [n] ^ pbuf [n] -I- 1 mod 2; 

8 // Case (b) n = 7 (mod 12). 

9 for(ui^l; 3 ui < X2 ; ui-l— k) 

10 for {u 2 ^ |"max(0, x\ — 3u^)^/^] ;U 2 <u\ && {n ^ 3u^ -I- u^) < X 2 ; U 2 ++) 

11 if (n mod 12 = 7) pbuf [n] ^pbuf [n] -k 1 mod 2; 

12 // Case (c) n = 11 (mod 12). 

13 for (ui ^ [(a;i/3)^/^] ; 2u\ < X 2 \ ui-k-k) 

14 for {u 2 ^ |"max(0, 3u^ — a; 2 )^/^] ',U 2 <U\ && (n ^ 3u^ — u^) > ; U 2 ++) 

15 if (n mod 12 = 11) pbuf [n] ^pbuf [n] -k 1 mod 2; 

16 // Sieve out numbers with square factors. 

17 for (g ^ 3; < a; 2 ; g-k-k) 

18 for (m ^ < S 2 ; m-k-k) 

19 pbuf [mg^] ^ 0 ; } 



The presentation of Algorithm 1 given here is quite different from that of 
Atkin and Bernstein. They avoid enumerating points and allocating storage for 
n divisible by small primes, and they use difference equations satisfied by the 
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quadratic forms to reduce the number of multiplications and square root oper- 
ations required. However, both versions have the same basic complexity. Up to 
O-constants, the operation count for Algorithm 1 is the sum of the number of 
lattice points enumerated, the number of scanlines required to find them, and 
the number of operations required by the squarefree sieve of lines 18-20. 




U2 




(c) n mod 12 = 11, n = 3uj — 



Fig. 1. The three cases of Theorem 1 and of Algorithms 1 and 2 
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1 /2 

It is easy to show that the squarefree sieve requires 0{x2 — xi + x^ ) op- 
erations, and that there are 0{x2 ) scanlines. The number of lattice points is 
closely related to the area of the swath they lie in — the number differing from 
the area by an amount which is Oix^ )• For each of the three cases, the area 
is 0{x2 — a^i). We conclude that the number of lattice points, and the total op- 
eration count for Algorithm 1, is 0{x2 — x\+ x^ )• When Algorithm 1 is used 
to sieve segments of length < i? in an interval of arbitrary length, the operation 
count for the entire interval is 0{{x2 — a^i)(l + -I- Thus, sieving 

with Algorithm 1 is inefficient if B is much less than x^ ■ (Atkin and Bernstein 
describe a further modification which reduces the operation count by a factor of 
lnlna; 2 , at the cost of slightly greater memory requirements.) 

We may refine the bound of Oix^ ) for the difference between the area 
of a swath and the number of points within it. The question of the minimal 
upper bound is closely related to the “circle problem” , which is concerned with 
estimating the difference between the number of lattice points within a circle of 
radius s/x and its area. By a result of van der Corput [19] it follows that for each 
case of Algorithm 1 the number of points enumerated is 0(x2 — a;i -I- x^ ). Van 
der Corput’s result generalizes earlier work of Voronoi on the Dirichlet divisor 
problem [20], and of Sierpihski on the circle problem [16]. (See Section 5 of 
this paper and the discussion following Theorem 2.4.2 in [7].) Furthermore, the 
key idea behind these results lets us reduce the number of scanlines needed to 
enumerate the points, giving our dissected sieve described below. 

3 Dissecting the Atkin-Bernstein Algorithm 

To reduce the number of scanlines, we dissect the swath into pieces, and then scan 
each piece in a direction roughly tangent to the boundary curves (see Figures 2 
and 3). We choose tangents with slopes defined by a Farey sequence of order r, 
and then use corresponding “cuts” to separate the pieces. The optimal choice 
for r is discussed below in Subsection 3.3. 



3.1 Notation and Background Material 

Before giving details of the dissection, we introduce some notation and state 
without proofs some properties of quadratic forms and of Farey sequences. 

We use vector notation, with vectors denoted by lowercase boldface letters, 
and matrices by uppercase boldface letters. The transpose of A is written A*. 

Definition 1. Given u = [ui U 2 ]* and a symmetric matrix A, let 

Qa{u) = u^Au = a\u\ + a2U\U2 + a^u^, 



where A and the coefficients Qj are related by 



A = 



ai 

02/2 



02/2 

03 
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■ VQAiu) = 0 



• VQa(«) = 0 



■ VQAiu) = 0 



Fig. 2. A dissection for Qa{u) = 3uf — using three cuts 



Lemma 1. Given a symmetric matrix A, then 

Qa{ru) = p^Qa{u), ( 1 ) 

Qa{'u + v) = Qa{u) + Qa{v) + 2v*-Au, ( 2 ) 

VQa{u) = 2Au. ( 3 ) 

Definition 2. Given a quadratic form Qa{u) and vector r, let the cutting line 
for T, or T-cut, be the set of points u at which the gradient 'VQa{u) is perpen- 
dicular to T, i.e., 



{u : T ■ VQa(u) = 0} = {tt : /« • m = 0} 

where n = At. We call n the coefficient vector for the cut. 

The r-cut is a line passing through the origin. It depends on the quadratic form 
Qa{u) as well as on r; the quadratic form should be clear from context. For 
a given x, the curve Qa{u) = a; is parallel to t where the curve intersects the 
T-cut. In fact, given a line k ■ u = 0 the vector r = A~^k is the associated 
tangent vector at the intersection of /« • m = 0, Qa(u) = x. 

We define the Farey sequence of order r, to be the ascending sequence of 
reduced fractions /3/a between 0 and 1, with denominators bounded by r: 

Tr = {/3/a : gcd(a,/3) = 1, 0</3<a< r}. 

Writing /3/a < fi' jo' for consecutive elements of we have 

a/3' — a' (3 = 1 (Theorem 28 in Hardy and Wright [6].) (4) 

Lemma 2 below is from The Art of Computer Programming [9, Exercise 1.3.2.18], 
and follows from [6, §3.4]. 
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Lemma 2 (Recursion for Farey Fractions). Let /3o/cro, /3i/oi, ■ ■ ■ denote 
the Farey sequence of order r. Then, 

l3o = 0, ao= 1; A = 1, oi = r; 

fdk+2 = L(^ + Ofc)/afc+iJ Pk+i - Pk] 

ctfc +2 = [(r + Qffc)/afc+iJ Ofc+i — otk- 



3.2 The Dissection Algorithm 

To dissect a swath, Algorithm 2 below uses a sequence of tangent vectors derived 
from P/a G tFr, of the form r = [-P ap in Cases (a) and (b), and t = [P ap 
in Case (c) of Theorem 1, so that in all cases the corresponding cutting lines, 
K ■ u = 0, run through the upper-right quadrant. For Case (c) we terminate 
the sequence upon reaching a cut of slope 1, i.e. P/a = 1/3, so we must choose 
r > 3. To dissect the swath over the entire quadrant in Case (b), we swap the 
roles of ui and U2 and then perform a similar dissection for Qa(u) = uf + Su^. 

The piece corresponding to consecutive tangents r and r' is the set of points 
u with xi < Qa(u) < X 2 and with u between the r-cut and the r'-cut. We 
exclude points on the x-cut and include points on the r'-cut. In Cases (a) and 
(c) of Theorem 1 the included points lying on the very last r'-cut, i.e., on the 
line U 2 = ui, lie outside the corresponding TZ. Similarly, in Case (b) the points on 
the line U 2 = 3ui are counted twice. Although this gives an incorrect calculation 
of P{n) mod 2 for n corresponding to these points, the end result is still correct 
because such n have square factors and are sieved out in the final pass. 

Algorithm 2 controls initialization, dissection into pieces, and calling the 
squarefree sieve routine; while Algorithm 3 (ScanPiece) scans a single piece, 
and Algorithm 4 (SquareFreeSieve) does the final sieving to eliminate square 
factors. We discuss each algorithm in turn. 

Algorithm 2 (SieveSegment: Dissected Sieve of Order r) Given r > 3 
and a preallocated hit vector pbuf as in Algorithm 1, this algorithm sets pbuf [n] 
such that upon completion we have pbuf[n] = 1 if and only if n is prime. 
SieveSegment (r, pbuf ) { 
assert 3 < pbuf . xl < pbuf . x2 ; 
assert r > 3; 

/3 <— 0 ; Of <— 1 ; 

/?' <— 1 ; a! ^ r\ 
while (TRUE) { 

// Case (a) n = l (mod 4) , handles n mod 12 G {1, 5, 9} . 

ScanPiece (1,4, [Ji], [-P a]*, [-P' a'Y , pbuf ) ; 

// Case (b) n = 7 (mod 12). 

ScanPiece (7,12, [qi], [~P a]*, [~P' a'Y , pbuf ) ; 

ScanPiece (7,12, io, \—P aP , — /?' a'P , pbuf ) ; 

if(3/3'<o') { // Case (c) n=ll(modl2). 
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13 ScanPiece (11,12 , [q _°] , [/3 a]*, [/?' a'Y , pbuf ) ; } 

14 

15 if (/?' = a') 

16 break ; 

17 // Advance to next Farey fraction of order r. 

18 k ^ [(r + a)/a '\ ; 

19 {/3, fc/3'-/3}; 

20 {a, Of'} ^ {o', fcof' — a}; 

21 } 

22 // Sieve out square factors , using Algorithm ~4 below. 

23 SquareFreeSieve ( pbuf ) ; } 

To scan a piece bounded by the cuts for tangents r, x'. Algorithm 3 trans- 
forms to another coordinate system, or “ti-space”. The coordinates are related 
by tt = Tv, with T =[t t'], so the map v — sends r to the unit horizon- 

tal vector and t' to the unit vertical vector. By the method used to construct r 
and t' from Farey fractions. Equation (4) implies det(T) = ±1, so the mapping 
is area-preserving and gives a one-one map between points in (see Figure 3). 

Working in ti-space. Algorithm 3 scans both horizontally and vertically, shift- 
ing between horizontal and vertical at the image of the “mediant-line” defined 
in Figure 3(a). We compute Qa(u) using 

Qa{u) = Qb{v) = hvj + b2ViV2 + 



with 



B = T^AT = 



Qa{t) 


K ■ t' 




'bi 


62/2 


K ■ t' 






62/2 


bs, 



( 5 ) 



Writing B = [h b'], the cutting lines and their mediant-line have images in 
t)-space satisfying 



b - V = Q for the r-cut 

b' ■ V = Q for the r'-cut 

{b + b') ■ V = 0 for the mediant-line. 

To bound the range of the scanlines, we use three “crossing points” illustrated 
in Figure 3(b), and defined in terms of “normalized crossing points” c, c', c" 
given by 

c = I det“^(r)|(aia36i)“^/^[62/2 - 6i]* 

c' = I det“^(r)|(aia363)“^/^[63 - 62/2]* 

c" = I det-i(T)|(aia3(6i + 62 + b3))-^/^[{b2/2 + 63) (-61 - 62/2)]*. 

(We allow for the possibility that det(T) yf ±1 in future implementations of Al- 
gorithm 3.) For diagonal forms Qa{u), these points are at intersections between 
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(a) Original coordinate system (w-space), 
showing the two cutting lines (solid) and 
the mediant-line (dashed) 




(b) Transformed coordinate system {v- 
space), showing crossing points (circled) 
used by Algorithm 3 



Fig. 3. A piece of a dissection, in two coordinate systems 
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Qb{v) = 1 and the images in i)-space of the x-cut, r'-cut, and mediant-line, 
respectively. By Equation ( 1 ), we see that y/xc lies at an intersection between 
Qb{v) = X and the image of the r-cut, and similarly for c', c". Given a;i > 0 , 
X2 > 0, we must have det(T)(a;2 — a^i) > 0 for the point to lie above 

ydrjc and to the left of . In order to maintain this relationship between 

the crossing points, Algorithm 3 swaps x\ and x^ when det(T) < 0 . 

Making use of the relationships described above. Algorithm 3 proceeds in 
much the same way as Algorithm 1 , although finding the bounding points for a 
scanline is more complicated because of the greater generality of the quadratics 
to be solved, and because both ends of the scanline may be bounded by either 
a conic or by a line. 

Algorithm 3 (ScanPiece: Process Lattice Points within a Piece) 

This routine enumerates all lattice points m G Z x Z lying within the piece 
corresponding to r and r'. Letting n = Qa{u) for each u enumerated, pbuf [n] 
is complemented if n mod m = k. 

1 ScanPiece ( fc, m. A, T, r', pbuf ) { 

2 T^[t r ']; 

3 if(det(T)>0) 

4 a;i <— pbuf . xl ; a;2 <— pbuf . x2 ; 

5 else 

6 a;i <— pbuf . x2 ; a;2^pbuf.xl; 

7 6i^Qa(t); 62 ^t'*At; bs^QAir'); ; 

8 c ^ I det“^(T)|(aia36i)“i/2[62/2 - bi]* ; 

9 c' ^ I det“^(T)|(aia363)“i/^[63 -62/2]*; 

10 c"^ |det-i(T)|(aia 3 ( 6 i + 62 + 63 ))-i/ 2 [( 62/2 + 63 ) (-61-62/2)]*; 

11 // Scan the half— piece below, or on, the mediant— line 

12 // (horizontal scanlines). 

13 for (u 2 ^ [[0 1] < L[o 1] ; ^^ 2 ++) { 

14 / ! d= discriminant of quadratic, or zero . 

15 d ^ max( 0 , 62^1 — 46 i (63^1 — a;i)) ; 

16 // "Cstart G R, "Cstop G R give limits for the scanline. 

17 Ustart ^ (-62U2 + v^)/( 26 i) ; 

18 d ^ 62^1 — 461(63 — S2) ; assert d>0; 

19 Ustop ^ min(-(263 + 62)^2/(261 + 62), (-62U2 + W)/( 26 i)) ; 

20 for (ui ^ 1 + [UstartJ ; Vi < Ustop ; Vi+ + ) { 

21 n ^ Qb{[vi V2 ]*) ; 

22 if(nmodm=fc) pbuf [n] ^pbuf[n] + 1 mod 2;}} 

23 // Scan the half— piece above, and off, the mediant— line 

24 //(vertical scanlines). 

25 for (ui ^ [[I 0] y/^c"] ; ui < [[1 0] y/SJc'J ; ui-P-P) { 

26 d ^ 62^1 — 463(61^1 — a;2) ; assert d>0; 

27 Ustart ^ max(-( 26 i + 62)^1/(263 + 62), (-62U1 - v^)/(263)) ; 

28 d <— max( 0 , 62^1 — 463(62^1 — a;i)) ; 

29 Ustop ^ (- 62 W 1 - Vd)/(2b3 ) ; 
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for ( W 2 ^ 1 + [WstartJ i ^2 < fstop i ^2+ + ) { 

n ^ Qb{[vi V2Y); 

if (n mod m = k) pbuf [n] ^pbuf[n] + 1 mod 2;}} 



Algorithm 4 is similar to the corresponding code of lines 18-20 in Algorithm 1, 
but uses the “Dirichlet hyperbola method” [18, Chapter 1.3, §3.2] to reduce the 
operation count from 0(a;2 — x\ + xy^) to 0(a;2 — x\+ xy^). 

Algorithm 4 (SquareFreeSieve: Sieve out Square Factors) 

This routine sets pbuf [n] = 0 for each n of the form n = mq^ , q > 1. 

SquareFreeSieve ( pbuf ) { 
a;i^pbuf.xl; ^2 <— pbuf . x 2 ; 
for(g^ 3 ; q<xy^;q + +) 
for {m ^ \xi/q‘^~\; mq^<X 2 ', m++) 
pbuf [mg^j ^ 0 ; 

for(m<— 1; m < xy^ ; m++) 
for (g ^ max(3, ; mq^<X 2 ', 9 + +) 

pbuf [mg^j ^ 0 ; } 



3.3 Timing Analysis and Optimal Order of Dissection 

Up to 0-constants, the operation count for Algorithm 2 is the sum of the num- 
bers of points, scanlines, pieces, and operations required by Algorithm 4. As 
mentioned above. Algorithm 4 requires 0(a;2 — a;i -I- x^ ) operations, and, by 
the work of van der Corput [19], the number of points is of the same order. 

For a given r, the number of pieces is O(r^) since the number of Farey frac- 
tions of order r is 3r^/7r^-|-0(rlnr). (See Theorem 330 in Hardy and Wright [6].) 
The number of pieces is dominated by the number of scanlines, which is shown 
in my thesis [5] to be 0(r^(a;2 — a;i)a ;2 + r'^ + xy^ fr). To roughly minimize 

the number of scanlines, we assume a ;2 > a^i and choose r to balance the and 
1/r terms in this bound. This gives r x {x 2 /{x 2 — and the number of 

scanlines is 

0(r^ -I- xy^ jr) = 0(a;2^^(a;2 - xi)~^/'^ + xy‘^{x 2 - xiY/y. 

1 /3 

The second term dominates provided X 2 — x\ Xrf , in which case the number 
of scanlines is 0 (xy'^(x 2 — 

Totaling these counts, we find, with r x {x 2 l{x 2 — that Algorithm 2 

requires 0{x2 — x\ + a^ 2 ^^) operations, provided X 2 — x\ ^ x\^^ . The same 
bound holds, with r x x\^^ , when X 2 — x\ x\^^ . When Algorithm 2 is used 
to sieve segments of length < H in an interval of arbitrary length, using the 
appropriate value of r for each segment, the operation count for the entire interval 
is 0((a;2 - a;i)(l + x\^^ jB) + 
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3.4 Implementation Notes 

The dissected sieve has been implemented as a program dsieve. Although this 
implementation is rudimentary, there are some optimizations. We reduce the size 
of the numbers used in Algorithm 3 (ScanPiece) by working both in t6-space 
and in a coordinate system in which u-space is re-centered around the lattice 
point given by rounding the components of c" to their nearest integers. We 
also get smaller numbers by computing Qa(u) — xi, rather than Qa(u) directly. 
As in the Atkin-Bernstein paper, ScanPiece uses Equation (2) to reduce the 
number of multiplications needed to update Qa(u) — xi as u varies. It reduces 
the number of square root operations by monitoring the values of Qa(u) — xi, 
K ■ It, and n' ■ u, to determine whether a point lies within a piece, and to decide 
when to move to the next scanline. 

Algorithm 4 (SquareFreeSieve) was modified to sieve only the odd numbers, 
and an additional parameter was added to control the point at which it makes 
the transition from the loop of lines 4-6 to the loop of lines 7-9. For dsieve, 
the optimal transition point was found experimentally to be near q = 1.5a;2^ . 
We also found experimentally, for a wide range of values of X 2 and B, that 
setting r = [0.5 -F 0.7(a;2/i?)^/^J approximately minimizes both the number of 
scanlines and the operation count. It should be noted that the constants 1.5 and 
0.7 are certainly dependent on the details of our implementation, and may also 
be machine dependent. 

3.5 Possible Improvements 

There are several ways in which dsieve could be improved. For simplicity in 
coding and analysis, we used a single Farey dissection. It may be more efficient to 
use three Farey dissections, each of an order chosen to optimize the corresponding 
case in Theorem 1. Also, “Farey-like” sequences tailored to each quadratic form 
may be more efficient — Sierpihski used a sequence of the form 

{P/a : gcd(a, P) = 1, + p"^ < r^} 

in his work on the circle problem [16]. Furthermore, besides the three pairs 
of congruence classes and quadratic forms used in Theorem 1, there are other 
choices — see the paper of Atkin and Bernstein for one example [1]. It seems to 
be an open question as to how to determine an optimal set of forms. 

Currently, dsieve enumerates too many points. Although it does not allocate 
storage for even indices, it does enumerate all points within a swath, including 
points yielding Qa(u) = 0 (mod 2). Reducing the number of points enumerated, 
and avoiding the costly test “n mod m = k” used in Algorithm 3 (ScanPiece) 
could improve the speed. However, this will not reduce the number of scanlines 
enumerated — which may dominate the operation count when X 2 — xi is small 
enough with respect to X 2 ' ■ Also, T (mod m) determines the periodic pattern of 
remainders taken by n = Qa{u) (mod m) as u moves along a scanline. Since the 
congruence class of T = [x r'] changes irregularly between calls to ScanPiece, 
it may be preferable to restrict T to a limited set of congruence classes. 
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We have not rigorously analyzed the size of numbers used by Algorithms 2, 3, 
and 4. The implementation dsieve uses a mixture of 32-bit and 64-bit integers, 
and 64-bit floating point numbers, and becomes unreliable near 10^®. 

4 Timing Comparisons 

Bernstein has implemented the Atkin-Bernstein algorithm and posted it on the 
web as a package of routines, “primegen 0 . 97”; see their paper for the URL [1]. 
Tables 1 and 2 show running times for primegen and for dsieve. Both programs 
were compiled to run on SUN SPARC computers, using the Gnu C-compiler 
(gcc) version 2.8.1, with compilation options -03 -mcpu=v8. Times were mea- 
sured on a 300 MHz UltraSPARC 5/10 with 64 megabytes of “main” memory. In 
addition, it has roughly 16 kilobytes of “level- 1” cache memory — very fast com- 
pared to main memory — and 512 kilobytes of somewhat slower level-2 cache. 
(The amounts and speeds of cache were estimated using a C implementation of 
the memld program given in [3, Appendix E].) 
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Table 1 . Time for primegen to count primes in the interval [xi, x\ + 10®], using 
B bits of memory 



The program primes . c provided in the primegen package was modified to 
print the count of primes in an interval [a;i, 0 : 2 ], and was run to count primes in 
several intervals. These counts were compared with those found by dsieve, and 
by a third program based on Robert Bennion’s “hopping sieve” [4]. Although 
Bernstein warns that the primegen code is not valid past x = 10^®, all pro- 
grams returned the same counts except for the interval [10^^, 10^^ -I- 10®], where 
primegen counts 10^^ -k 111377247 = 7 • 119522861® and 10^^ -k 158245891 = 
11 • 95346259® as primes. 

The buffer size used by primegen {B, in our notation) is set at compile 
time. Table 1 shows running times for primegen to count primes in the interval 
[xi,X 2 = a;i -k 10®] for several combinations of B and a:i. In Bernstein’s installa- 
tion instructions he suggests choosing B so that data used by the inner loop of 
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the algorithm resides in level-1 cache. (The inner loop treats n = QQk+d for fixed 
d, where d takes one of the 16 values relatively prime to 60.) For the UltraSPARC 
computer, we used the suggested value of i? = 16 • 128128 « 2.05 • 10®. 

To avoid having a runtime that became linear in a;i for very large xi (lin- 
ear with a small O-constant), we modified the routine primegen_skipto to use 
division instead of repeated subtraction in its calculation of a quotient. 

As well as showing that primegen slows as grows larger than B, Table 1 
illustrates that the operation count may only roughly predict the running time 
on a computer with cache memory. Increasing B reduces the operation count 
for sieving an interval, but also increases the chance that memory references 
will miss the level- 1 cache. This slowdown as the locality of memory references 
decreases can be striking. On the computer used for these tests, widely scattered 
references to “main” memory were measured to be roughly 20 times slower than 
references to level-1 cache. (An informative discussion of cache memory is given 
in [3, Chapter 3].) 
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Table 2. Time for dsieve to count primes in the interval [xi, X 2 = xi + 10®], 
using « B bits of memory and a Farey dissection of order r « 0.7{x2/BY^^ . 
The “sqfree” column gives the time required to sieve out square factors 



Table 2 shows running times for dsieve to count primes in the interval 
[x\,X 2 = xi + 10®], using two different values of B, with B depending on X 2 - 

1 /3 

The entries with B « 10x2^ illustrate the running time when using a “small” 
amount of memory, while the entries with B « X 2 show the running time with 
memory usage comparable to that needed for efficient operation of previously 
known sieves. In both cases and for all values of xi, after computing pbuf it 
took roughly 18 seconds to count the primes (1-bits in pbuf). As expected, the 
running time does not greatly increase as increases. The slowdown for larger 
is presumably due in part to decreasing locality of reference, although more 
detailed statistics on operation counts should be collected in order to better 
understand these results. 
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5 Miscellaneous Remarks 

The ideas of this paper can also be used with Qa{u) = U\U 2 , giving a dissected 
“Eratosthenes-like” sieve. This corresponds to the Dirichlet divisor problem, 
which is concerned with estimating the number, D{x), of lattice points within 
the hyperbola U\U 2 <x,u\> 0, U 2 > 0. Voronoi [20] used a dissection based on 
the Farey-like sequence 



{(3/a : gcd(a,/3) = 1, a/3 < t}, 

with t = to show that D{x) = a;ln(a;) -I- (2y — l)a; -I- 0(a;^/^lna;), where 

7 « 0.5772 ... is Euler’s constant. His result was an improvement of an earlier 
result of Dirichlet, who used the “hyperbola method” to get an error term of 
0{x^^^) instead of 0{x^^^~^'^) for the approximation of D{x). Voronoi’s result for 
the Dirichlet divisor problem suggests that a dissected Eratosthenes-like sieve 
would require 0(xl^^~^'^) bits and 0 {x 2 (x 2 — a;i -I- x^^^)) operations to sieve the 
interval [x\,X 2 \- 

We can also use dissection to improve some factoring algorithms. For exam- 
ple, trial division searches for a solution to n = Qa{u) = U\U 2 , and dissection 
would reduce the operation count to Dissection would similarly re- 

duce the number of operations needed to solve the quadratics used by D. H. and 
Emma Lehmer [12] for factoring. 

Sierpihski’s bound has since been improved to an bound 

for the difference between the number of lattice points within a circle of radius 
^/x and its area [8, § 13.8], and the conjectured bound is The im- 

proved bound was proven using analytic techniques, and it is not clear if these 
techniques could be applied to making sieving more efficient. However, the re- 
sult does raise the intriguing speculation that we may be able to sieve efficiently 
using significantly less than Oix^ ) bits of memory. 
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Abstract. We describe some algorithms for computing the cardinality 
of hyperelliptic curves and their Jacobians over hnite fields. They include 
several methods for obtaining the result modulo small primes and prime 
powers, in particular an algorithm a la Schoof for genus 2 using Can- 
tor’s division polynomials. These are combined with a birthday paradox 
algorithm to calculate the cardinality. Our methods are practical and 
we give actual results computed using our current implementation. The 
Jacobian groups we handle are larger than those previously reported in 
the literature. 



Introduction 

In recent years there has been a surge of interest in algorithmic aspects of curves. 
When presented with any curve, a natural task is to compute the number of 
points on it with coordinates in some finite field. When the finite field is large 
this is generally difficult to do. 

Rene Schoof gave a polynomial time algorithm for counting points on elliptic 
curves i.e., those of genus 1, in his ground-breaking paper [Sch85]. Subsequent 
improvements by Elkies and Atkin ([Sch95], [Mor95], [Elk98]) lowered the expo- 
nent to the point where efficient implementations became possible. After further 
improvements ([Cou96], [Ler97]) several implementations of the Schoof-Elkies- 
Atkin algorithm were actually written and very large finite fields can now be 
handled in practice ([Mor95], [Ver99]). 

For higher genus, significant theoretical progress was made by Pila who gave 
a polynomial time algorithm in [Pil90] (see also [HI98]). However to date these 
methods have not been developed as extensively as the elliptic case. As a first step 
towards closing this gap it is fruitful to concentrate on low genus hyperelliptic 
curves, as these are a natural first generalization of elliptic curves and techniques 
used in the elliptic case can be adapted. Such techniques include Schoof-like 
methods and several others which all contribute to a practical algorithm. 

We mention two possible applications of the ability to count points on low 
genus hyperelliptic curves. An early theoretical application was the proof that 
primality testing is in probabilistic polynomial time, [AH92]. A practical ap- 
plication results from the apparent difficulty of computing discrete logarithms 
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in the Jacobian groups of these curves. In low genus, no sub-exponential algo- 
rithms are currently known, except for some very thin sets of examples ([Riic99], 
[FR94]) and hence the Jacobian group of a random curve is likely to be suitable 
for constructing cryptosystems [Kob89]. To build such a cryptosystem, it is first 
desirable to check that the group order has a large prime factor since otherwise 
the logarithm could be computed in small subgroups [PH78]. 

We restrict ourselves to odd characteristic for simplicity. We will work with 
models of odd degree where arithmetic is analogous to that of imaginary qua- 
dratic fields. For the even degree alternative, which is similar to real quadratic 
fields, see the recent paper [ST99] which describes a birthday paradox algorithm 
optimized using an analogue of Shanks’ infrastructure. 

Our contribution contains several complementary approaches to the problem 
of finding the size of Jacobian groups, all of which have been implemented. 
By combining these approaches we have been able to count larger groups than 
previously reported in the literature. 

The first approach is an efficient birthday paradox algorithm for hyperelliptic 
curves. We have filled in all the details required for a large-scale distributed 
implementation, although the basic idea has been known for 20 years. In our 
implementation we also use an optimized group operation for genus 2, in which 
we have reduced the number of field operations required. 

The time taken grows as a small power of the field size and this algorithm, 
if used in isolation, would take a prohibitive amount of time to handle large 
groups such as those of cryptographic size. However our version of it can take 
advantage of prior information on the result modulo some integer. We elaborate 
various strategies for collecting as much of this information as possible. 

We show that when the characteristic p is not too large, the result modulo p 
can be obtained surprisingly easily using the Cartier-Manin operator. It provides 
an elegant and self-contained method based on theoretical material proved in the 
1960’s. 

To go further, we also extend Schoof’s algorithm to genus 2 curves using 
Cantor’s division polynomials. On the basis of previous outlines existing in the 
literature, but not directly implementable, we elaborated a practical algorithm 
and programmed it in Magma. For the case where the modulus is a power of 2, 
we are able to bypass computations with division polynomials and use a much 
faster technique based on formulae for halving in the Jacobian. 

The combinations of these techniques has allowed us to count genus 2 groups 
with as many as 10^® elements. 

We would particularly like to thank Eric Schost of the GAGE laboratory 
at Ecole Polytechnique for helpful discussion concerning algebraic systems. Fur- 
thermore his assistance in computing Grobner bases was invaluable and allowed 
us to compute group orders modulo larger powers of 2 than would otherwise 
have been possible. 

We also thank Frangois Morain for many constructive comments on this 
paper. 
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Prerequisites and Notations 

We will take a concrete approach, concentrating on arithmetic and algorithmic 
aspects rather than more abstract geometric ones. 

Let g be a positive integer and let be the finite field of g = p” elements, 
where p is an odd prime. For our purposes, a hyperelliptic curve of genus g is 
the set of solutions {x,y) of the equation y^ = f{x), where f{x) is a monic 
polynomial of degree 2g+l with coefficients in and with distinct roots^. Note 
that the coordinates may be in the base field F^ or in an extension field. 

When a point P = {xp,yp) is on the curve C, its opposite is the point 
—P = (xp,—yp). A divisor^ is a formal sum D — ^^^Pi of points on C. Note 
that points may be repeated with some multiplicity in the sum. A semi-reduced 
divisor is a divisor with no two points opposite. Such a divisor with k points 
is said to have weight k. A reduced divisor is a semi-reduced divisor of weight 
k< g. 

The Jacobian, denoted J, is the set of reduced divisors. An important fact 
is that one can define an addition operation on reduced divisors which makes J 
into a group, whereas this is not possible on the curve itself directly. This group 
law is denoted by -I- and will be described in the next section. 

A convenient representation of reduced (and semi-reduced) divisors, due to 
Mumford [Mum84], uses a pair of polynomials {u{x),v{x)). Here u{x) = rii(^ ~ 
Xi) and v{x) interpolates the points Pi respecting multiplicities. More precisely 
u = 0 or deg v < deg u, and u divides f — v^. We say that a semi-reduced divisor 
is defined over a field F when the coefficients of u and v are in F (even though 
the coordinates xt and yi may be in an extension field) and write J /F for the 
set of such divisors. 

Most reduced divisors have weight g. The set of those with strictly lower 
weight is called 0. A divisor of weight 1 i.e., with a single point P = {xp, yp), 
is represented by (u(x), v(x)} = (x — xp,yp). The unique divisor of weight 0, 
O = {u{x),v{x)) = (1,0), is the neutral element of the addition law. Scalar 
multiplication by an integer I is denoted by: 

[l]D = D D + ■ ■ ■ D . (^ times) (1) 

We say that D is an ^-torsion divisor whenever [l]D = O. The set of all ^-torsion 
divisors, including those defined over extension fields, is denoted by J [^] . 

We concentrate particularly on genus-2 curves and in this case the divisors 
in J \ 6* have the form: 



D = {x^ U\X U2,VoX Vi) . (2) 

^ Strictly speaking, this is the affine part of a smooth projective model. In genus 2 
every curve is birationally equivalent to such a curve provided the base field is large 
enough. 

^ Strictly speaking, these are degree-0 divisors with the multiplicity of the point at 
infinity left implicit. 
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1 Group Law in the Jacobian 

We will sketch the group law i.e., addition of reduced divisors, using the intuitive 
‘sum of points’ notation and then describe efficient formulae for computing the 
law in genus 2 using Mumford’s representation. 

The computation of Di + D 2 can be viewed, at a high level of abstraction, 
as the following three steps: 

— form a (non-reduced) divisor with all the points of D\ and D 2 , 

— semi-reduce it by eliminating all pairs of opposite points, 

— reduce it completely. 

The third step is the only one that presents any difficulty^. When we reach 
it we have a semi-reduced divisor with at most 2g points. If there are g or 
fewer then no reduction is necessary but if there are more than g we reduce by a 
higher-genus analogue of the well known chord-and-tangent operation for elliptic 
curves. 



1.1 Reduction Step 

Fix g = 2 and, for the moment, consider a semi-reduced divisor R with 3 distinct 
points. The reduction of R is as follows. 

Let y = a(x) be the equation of the parabola (or perhaps line) interpolating 
the three points. The roots of / — are the abscissae of the intersections be- 
tween the parabola and the curve. This is a quintic polynomial so there are five 
intersections (including multiplicities). We already know 3 of them, the points 
of R. Form a divisor S with the other two, and the result of the reduction is —S. 

In the more frequent case where R has 4 points, choose an interpolating cubic 
(or lower degree) polynomial a(x) instead. Then f — has degree 5 or 6 and 
we know 4 intersections. Form S with the others and the result is —S. 

In cases where some points of R are repeated, the interpolation step is ad- 
justed to ensure tangency to the curve with sufficient multiplicity. Also, in genus 
g > 2 the reduction step may need to be repeated several times. 

In practice it would be inefficient to compute the group law this way using the 
representation of divisors as sums of points, since the individual points may be 
defined over extension fields. By using Mumford’s notation we can work entirely 
in the field of definition of the divisors. 

1.2 Group Law in Mumford’s Notation 

Cantor gave two forms of the group law using Mumford’s notation in [Can87]. 
One was a direct analogue of Gauss’s reduction of binary quadratic forms of 

® In a more classical treatment the reduction would be described as choosing a rep- 
resentative for an equivalence class of degree 0 divisors modulo linear equivalence, 
where linearly equivalent divisors are those that differ by a principal divisor. 
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negative discriminant, the other an asymptotically fast algorithm for high genus 
making clever use of fast polynomial arithmetic. 

We describe an efficient algorithm, carefully optimized to reduce the number 
of operations required. We find that in genus 2 doubling a divisor or adding 
two divisors both take 30 multiplication operations and 2 inversions, in general. 
Note for comparison that optimized elliptic curve operations typically take 3 or 
4 multiplications and 1 inversion. 

Space limits us to a brief description of the genus 2 doubling operation. Let 
D = (u, v). We cover the cases that may occur, in order of increasing complexity. 

Simple case: If u = 0 the result is simply O. 

Weight 1: Here u{x) = x — xp and v(x) = yp. The result is {{x — xp^ ,ax + b), 
where aa; + 5 is the tangent line at P with a = f {xp)/2yp and b = yp — axp. 

Weight 2: Compute the resultant r = U 2 Vq + vf — uiVqVi of u and v. 

Resultant 0: If r = 0 then u and v have a root in common i.e., D has 
a point with ordinate 0. Isolate the other point by a;p = —ui + vi/vq, 
yp = vi + vqXp and return to the weight 1 case above. 

General case: Consider the fact that p is a square root of / modulo 
u. We can double the multiplicity of all points in D by using a Newton 
iteration to compute a square root modulo vF'. 

— Newton iteration: set U = u^, and V = {v + j Iv)j2 mod U, 

— Get ‘other’ roots: set U = {f — V^)/U, 

— Make U monic, 

— Reduce V modulo U, 

The result is {U, —V). 

Several observations help to optimize calculation with these formulae: after the 
first step, V = v mod u; also the division by U in the second step is exact; not 
all coefficients of the polynomials are really needed; finally some multiplications 
can be avoided using Karatsuba’s algorithm. 

The general addition operation is similar to doubling although the Newton 
iteration is replaced by a little Chinese Remainder calculation and more cases 
need to be handled. Since the details are somewhat tedious, we give the resulting 
pseudo-code and sample C code at the following Web site: 
http : / / cristal . inria.fr/~harley/hyper/ 

2 Probenius Endomorphism 

In this section we collect some useful results and quote them without proof. A 
starting point for the reader interested in pursuing this material is [IR82] and 
the references therein. 

We first describe properties of the g-power Frobenius endomorphism (j){x) = 
x'^. Note that it has no effect on elements of but it becomes non-trivial in 
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extension fields. This map extends naturally to points, by transforming their x 
and y coordinates. It extends further to divisors by acting point- wise. 

Crucially, this latter action is equivalent to acting on each coefficient of the 
u and V polynomials in Mumford’s notation. When a divisor is defined over F^, 
(j) may permute its points but it leaves the divisor as a whole invariant. 



2.1 Characteristic Polynomial 

The (j) operator acts linearly and has a characteristic polynomial of degree 2g 
with integer coefficients. In genus 2 it is known to have the form: 

Xit) = t"^- sit^ + S 2 t^ - siqt + , (3) 

so that x(</') is the identity map on all of J, in other words: 

VP G J, - [si](/."(P) + [S 2 ]<I>\P) - [sig]</-(P) + [q^]P = 0 . (4) 

The so-called Riemann hypothesis for curves, on the roots of their zeta func- 
tions, was proved by Weil and implies that the complex roots of x have absolute 
value y/q. Hence, in genus 2 the following bounds apply: |si | < and |s2 1 < bg. 

2.2 Relations Between Frobenins and Cardinalities 

The Frobenius is intimately related to the number of points on the curve and 
the number of divisors in J, over the base field and its extensions. 

First of all, knowledge of x is equivalent to that of #C/F^i for 1 < i < 5. In 
genus 2 the following formulae relate them: 

ffC/'Wq = q — s\ and #C/Fg2 = q^ — s\+ 2s2 ■ (5) 

Furthermore /¥q is completely determined by x according to the formula 
#J /¥q = x(l)- An important consequence is that the group order is constrained 
to a rather small interval, the Hasse-Weil interval: 

<#J/F,< . (6) 

In the reverse direction, knowledge of /¥q almost determines x for q large 
enough. For instance in genus 2, (#J /Wq) — q^ — 1 = S2 — si (g-b 1) and the bound 
on S2 given above ensures that there are 0(1) possibilities. 

3 Birthday Paradox Algorithm 

To compute the group order N = /¥q exactly we search for it in the Hasse- 

Weil interval which has width w close to 4g(7®“^fo. The first few coefficients Si 
of X can be computed by exhaustively counting points on the curve over F^^. 
Doing so for i < I reduces the search interval to width w = but 

costs 0{q^) (see [Elk98]). In genus 2 this is not useful and one simply takes 
w = 2[^{q+\)^\. 
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3.1 Computing the Order of the Group 

Assume for the moment that we know how to compute the order n of a randomly 
chosen divisor D in J /F^ (from now on the term “divisor” always refers to a 
reduced divisor). Writing e for the group exponent, we have n \ e and e | N and 
thus N is restricted to at most [(w + l)/n] possibilities. Usually n> w and so 
N is completely determined. 

It is possible for n to be smaller, though. In such a case we could try several 
other randomly chosen divisors, taking n to be the least common multiple of 
their orders and stopping \i n > w. After a few tries n will converge to e and if 
e > w the method terminates. 

However in rare cases the exponent itself may be small, e < w. It is known 
that J /Wq is the product of at most 2g cyclic groups and thus e > ^ — 1 and 
in fact this lower bound can be attained. 

It is possible to obtain further information by determining the orders of 
divisors in the Jacobian group of the quadratic twist curve, but even this may 
not be sufficient. We do not yet have a completely satisfactory solution for such 
a rare case, however we mention that the Weil pairing may provide one. 

3.2 Couiputiug the Order of Oue Divisor 

To determine the order n of an arbitrary divisor D we find some multiple of n, 
factor it and search for its smallest factor d such that [d]D = O. 

There are certainly multiples of n in the search interval (since the group 
order is one such) and we can find one of them using a birthday paradox al- 
gorithm, in particular a distributed version of Pollard’s lambda method [Pol78] 
with distinguished points. For a similar Pollard rho method see [vOW99]. 

Since the width of the search interval is w, we expect to determine the mul- 
tiple after 0{\/w) operations in the Jacobian. By using distinguished points and 
distributing the computation on M machines, this takes negligible space and 
0{^/w([ogqY /M) time^. 

The birthday paradox algorithm is as follows. 

— Choose some distinguishing characteristic. 

— Choose a hash function h that hashes divisors to the range 0..19, say. 

— Pick 20 random step lengths k > 0 with average roughly M^/w, 

— Precompute the 20 divisors Di = \li]D. 

— Precompute E = \c\D. 

Here c is the center of the search interval. In genus 2 c = + &q+ 1. The 

calculation then consists of many ‘chains’ of iterations run on M client machines: 

— Pick a random r < w and compute R = [r]D. 

— Pick a random bit b and if it is 1 set i? to i? J- U . 

— While R is not distinguished, set r := r + lh(R) and R:= R+ 

— Store the distinguished i? on a central server along with r and b. 



^ Using classical algorithms for field arithmetic. 
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The distinguishing feature must be chosen to occur with probability significantly 
less than ^Jw/M, say 50 times less. Thus each chain takes about ^/w/M/50 steps 
and has length about w/50. 

Note that chains with 6 = 0 visit many pseudo-random divisors in the set 

= {[r]D I 0 < r < w} and a few with larger r. Chains with 6=1 visit many 
divisors in 52 = {E + [r]D \ 0 < r < w} and a few with larger r. However the 
choice of E guarantees that the intersection I = Si C\ S 2 contains at least w/2 
divisors. 

Now after a total of 0{ySw) steps have been performed, 0{ySju) divisors have 
been visited and 0{^/w) of them are expected to be in /. Then the birthday 
paradox guarantees a significant chance that a useful collision occurs i.e., that 
the same divisor R is visited twice with different bits 6. Shortly afterwards a 
useful collision of distinguished points is detected at the server, between Rq and 
Ri say. 

Therefore tq = c + ri modulo n and finally c -I- ri — ro is the desired multiple 
of n. 



3.3 Beyond the Birthday Paradox 

To handle larger examples than is possible with the birthday paradox algorithm 
alone, we precompute the Jacobian order modulo some integer. If N is known 
modulo m then the search for a multiple of a divisor’s order can be restricted to 
an arithmetic progression modulo m, rather than the entire search interval®. In 
this way the expected number of operations can be reduced by a factor y/m. 

The algorithm outlined above needs to be modified as follows (we can assume 
that m is much smaller than w since otherwise no birthday paradox algorithm 
would be required!). 

— Increase the frequency of the distinguishing characteristic by a factor -y/m. 

— The step lengths must be multiples of m chosen with average length M ^fwm. 

— Replace E with [z] D where z is nearest c such that z = N mod m. 

To compute N modulo m with m as large as possible, we will first compute 
it modulo small primes and prime powers using various techniques explained in 
the next few sections. Then the Chinese Remainder Theorem gives N modulo 
their product. 

To date this use of local information has speeded up the birthday paradox al- 
gorithm by a significant factor in practice. It should be pointed out however that 
while the birthday paradox algorithm takes exponential time, the Schoof-like al- 
gorithm described below takes polynomial time. Hence it can be expected that 
for future calculations with very large Jacobians, the Schoof part will provide 
most of the information. 

® Note that we could also take advantage of partial information that restricted N to 
several arithmetic progressions modulo m. 
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4 Cartier-Manin Operator and Hasse-Witt Matrix 

We propose a method for calculating the order of the Jacobian modulo the 
characteristic p of the base field, by using the so-called Cartier-Manin operator 
and its concrete representation as the Hasse-Witt matrix (see [Car57]). In the 
case of hyperelliptic curves, this g x g matrix can be computed by a method 
given in [Yui78] which generalizes the computation of the Hasse invariant for 
elliptic curves. Yui’s result is as follows: 

Theorem 1. Let = f{x) with deg f = 2g-\-l he the equation of a genus g hy- 
perelliptic curve. Denote by Ci the coefficient of in the polynomial 
Then the Hasse- Witt matrix is given by 

A = . (7) 

In [Man65] , Manin relates it to the characteristic polynomial of the Frobenius 
modulo p. For a matrix A = (oy), let A^'A denote the elementwise p-th power 
i.e., (ofj). Then Manin proved the following result: 

Theorem 2. Let C he a curve of genus g defined over a finite field Fp» . Let A be 
the Hasse-Witt matrix ofC, and let A^ = AA^A ■ ■ ■ A^"p K Let K{t) be the char- 
acteristic polynomial of the matrix A^, and x(t) the characteristic polynomial of 
the Frobenius endomorphism. Then 

X{t) = {-iyCK{t) modp . (8) 

Now it is straightforward to compute x(t) modulo the characteristic p and 
hence #J/Fqmodp, provided that p is not too large (say at most 100000). 
Note that this is a very efficient way to get information on the Jacobian order, 
particularly when p is moderately large. Such a situation can occur in practice 
with fields chosen, for implementation reasons, to be of the form Fpn with p close 
to a power of 2 such as p = 2® — 5 or p = 2^® — 15. 

5 Algorithm d la Schoof 

In this section we describe a polynomial time algorithm d la Schoof for computing 
the cardinality of J/F^ in genus 2. This algorithm follows theoretical work of 
Pila [Pil90] and Kampkotter [Kam91]. We make extensive use of the division 
polynomials described by Cantor [Can94] . 

5.1 Hyperelliptic Analogue of Schoof’s Algorithm 

The hyperelliptic analogue of Schoof’s algorithm consists of computing x mod- 
ulo some small primes I by working in J[^]. Once this has been done, modulo 
sufficiently many primes (or prime powers), then x can be recovered exactly by 
the Chinese Remainder Theorem. From the bounds on Si above, it suffices to 
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consider I = 0 (log( 7 ). In practice we use a few small I, determine x modulo 
their product, and use this information to optimize a birthday paradox search 
as described previously. 

Let I be a prime power co-prime with the characteristic. Then the subgroup 
of ^-torsion points has the structure J[^] = (Z/^Z)^®. Moreover, the Frobenius 
acts linearly on this subgroup and Tate’s theorem [Tat 66 ] states that the charac- 
teristic polynomial of the induced endomorphism is precisely the characteristic 
polynomial of the Frobenius endomorphism on J with its coefficients reduced 
modulo 1. Hence by computing the elements of J[l] and the Frobenius action on 
them, we can get the characteristic polynomial modulo 1. 

The following lemma due to Kampkotter simplifies the problem. 

Lemma 1. If I is an odd prime power, then the set 3\0 contains a Z/lZ-basis 
ofJ[l]. 

Thus the Frobenius endomorphism on J [^] is completely determined by its action 
on J[^] \ 0. 

Let D = {x^ + u\x + U 2 , vqx + vi) be a divisor in J \ 6 >, then the condition 
[l]D = O can be expressed by a finite set of rational equations in u\, U 2 , vq, v\. 
More precisely, there exists an ideal // of the polynomial ring ¥g[Ui, U2,Vq, Vi] 
such that D lies in J[^] \ 6 > if and only if f{ui , U 2 , vq, ui) = 0 for all polynomials 
/ in (a generating set of) the ideal //. In [Kam9I], Kampkotter gives explicit 
formulae for multivariate polynomials generating //. 

From now on, we can represent a generic element of J [^] \ 6 * by the quotient 
ring ¥q[Ui, U 2 , Vq, Vi]///. The Frobenius action can be computed for this element 
and it is possible to find its minimal polynomial by brute force. The characteristic 
polynomial is then easy to recover (at least in the case where I is a prime) 
and we are done. This method due to Pila and Kampkotter has polynomial- 
time complexity, however it involves arithmetic on ideals which requires time- 
consuming computations of Grdbner bases. In the following we propose another 
method which avoids the use of ideals. 



5.2 Cantor’s Division Polynomials 



In [Can94] , Cantor defined division polynomials of hyperelliptic curves, general- 
izing the elliptic case, and gave an efficient recursion to build them. 

These polynomials are closely related to Kampkdtter’s ideal //, but they allow 
a Schoof-like algorithm to work mostly with one instead of four variables. An 
approximate interpretation of the phenomenon is that the division polynomials 
lead to a representation of // directly computed in a convenient form (almost a 
Grobner basis for a lexicographical order) . 

Cantor’s construction provides 6 sequences of polynomials dp ^ , d® , d 2 ^ and 
Cq \ ef \ 62 ^ such that for divisors P = {x — xp,yp) of weight 1 in general 
position, we get 



[l]P 




di\xp) df{xp) f e^l\xp) e^^^(xp) \\ 

do\xp) d!i\xp)' ^!'q\xp)} j 



(9) 
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The degrees of these division polynomials are 



do 


di 


d.2 


eo 


ei 


62 


2E - 1 


2E - 2 


2E - 3 


- 2 


SE - 2 


SE -3 



By lemma 1 it is sufficient to consider divisors D ^ O. In order to multiply 
D = {u{x),v{x)) by I we express it as a sum of two divisors of weight 1 i.e., 
we write D = P\ + P 2 - These divisors are given by Pi = {x — x\,yi) and 
P 2 = {x — X 2 , j/ 2 ) where xi and X 2 are the roots of u{x) and yi = v{xi). Clearly 
[i]D = [;]Pi + [i]P2. 

The divisor D is an Ptorsion divisor if and only if [^] P\ and [I] P 2 are opposite 
divisors. This last condition is converted into a condition on the polynomial 
representations [^]Pi = (up^ (x), (a;)) and [l]P 2 = {up 2 (x),vp^{x)). Indeed two 

divisors are opposite if their u polynomials are equal and their v polynomials 
are opposite. Hence the elements of J[l] \ 6> are characterized by a set of rational 
equations in the 4 indeterminates xi,X 2 ,yi,y 2 , two of them involving only the 
two indeterminates xi and X 2 - 

Thus we get an ideal similar to // represented in a convenient form: we can 
eliminate X 2 with the two bivariate equations by computing some resultants, then 
we have a univariate polynomial in xi and for each root xi it is not difficult to 
recover the corresponding values of X 2 , yi and j/ 2 - 



5.3 Details of the Algorithm 

Next we explain the computation of the characteristic polynomial modulo a fixed 
prime power 1. Here we will assume that I is odd (the even case discussed in the 
next section). 



Building an Elimination Polynomial for x^. We first compute Cantor’s 
Pdivision polynomials. We refer to the original paper [Can94] for the recursion 
formulae and the proof of the construction. This phase takes negligible time 
compared to what follows. 

The second step is to eliminate X 2 in the two bivariate equations. The system 
looks like 



( Ei{xi,X2) = di{xi)d2{x2) - di{x2)d2{xi) = 0 , , . 

\E2{xi,X 2) = do{xi)d2ix2) - do{x2)d2ixi) = 0 . 

The polynomial (xi — X 2 ) is clearly a common factor of Ei and E 2 , and this 
factor is a parasite: it does not lead to a ^-torsion divisor®. We throw away this 
factor and consider the new reduced system, still denoting the two equations by 



If there is another common factor of and E 2 , we have to throw it away. This occurs 
when a non trivial /-torsion divisor is in 0. The values for the degrees assume that 
we are in the generic case. 
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Ei{xi,X2) and E2{xi,X2) ■ Then we eliminate X2 by computing the following 
resultant 



R{xi) =ReSx^{Ei{xi,X2),E2{xi,X2)) =Q ■ ( 11 ) 

We can then note that R{x\) is divisible by some high power of ^2(3^1)- Indeed, 
if ^2(3^1) = 0 then the expressions E\ and E2 have common roots (at the roots 
of d2{x2))- The power of d2 in R is S = 2 P — 2 . We assume that the base field 
is large enough and we specialize the system at many distinct values for xi. 
Substituting for xi, the system becomes two univariate polynomials in X2^ 
for which we compute the resultant rj. With enough pairs (^i, r^) i.e., one more 
than a bound on the degree of R{x\) = R{x\) / {d2{xi)Y ^ we can recover R{xi) 
by interpolation. Knowing the degrees of do,di,d2, it is easy to get 

deg R{xi) = Al^ - IQf + 6 . ( 12 ) 



Eliminating the Parasites (Optional). As previously mentioned there are 
divisors of ^-torsion and thus the degree of R{xi) is too high by a factor 4 . 
This means that there are still a lot of parasite factors, due to the fact that we 
only took conditions on the abscissae 3 ;i,a ;2 into account and imposed nothing 
on the ordinates yi,j/2- Two strategies can be used: we can decide to live with 
these parasites and go on to the next step or we can compute another resultant 
to eliminate them (and get a polynomial of degree — 1 ). The choice depends on 
the relative speeds of the resultant computation and the root-finding algorithm. 

In order to eliminate the parasites we construct a third equation E^{x\, X2), 
coming from the fact that the ordinates of [^]Pi and [^]P2 are opposite. We write 
that the coefficients are opposite. 



m 



ei(xi) _ 61 ( 3 : 2 ) 

eo(xi) "^ 60 ( 2 : 2 ) 

e2(a;i) _ _ 62(2:2) 

60(2:1) ^^60(2:2) 



( 13 ) 



and this system implies that E^{xi,X2) = ei(xi)e2(3;2) — 61(0:2)62 (3:1) = 0. 

Taking the resultant between E\ and E3, we get a polynomial ^(a;!) of degree 
12 ^"^ — 30 ^^ -I- 18 whose GCD with R{x\) is of degree — 1 (in general, a few 
parasites may remain in rare cases). We still denote this GCD by i?(3;i) for 
convenience. 



Recovering the Result Modulo 1 . To find the result we factor i?( 3 ;i) and, 
for each irreducible factor, we construct an extension of using this factor to 
get a root Xi of R{xi). Then we substitute this root into Ei and E2 and recover 
the corresponding root X2 ■ Using the equation of the curve we get the ordinates 
Yi and Y2, which may be in a quadratic extension. We get the two divisors 
P\ = {x — Ai, Yi) and P2 = {x — ^2,^2) and check whether [l]{Pi + P2) = O or 
[l](Pi — P2) = O. If neither holds, then we started from a parasite solution and 
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we try another factor of R{xi). In the favorable case we get an ^-torsion divisor 
D with which we check the Frobenius equation. To do so we compute 

[si]4>^{D) + [qsi mod 1]4>{D) , (14) 

for every si G [0,1— 1] and 

+ [s 2 ](j}'^{D) + [q"^ mod l]D , (15) 

for every S 2 G [0,^ — 1]. We only keep the pairs (si , S 2 ) for which these are equal. 

If there is only one pair (si, S 2 ) left, or if there are several pairs all leading to 
the same value for the cardinality modulo I, then it is not necessary to continue 
with another factor. Thus it is usually not necessary to have a complete fac- 
torization of R{xi) and the computation is faster if one starts with irreducible 
factors of smallest degree. 

We summarize the above in the following: 



Algorithm. Computation of #J/Fg modulo 1 . 

1. Compute i?(xi). 

2. Find a factor of R{xi) of smallest degree. 

3. Build Pi and P 2 with this factor. 

4. Check if Pi + P 2 or Pi — P 2 is an ^-torsion divisor. If so call it D, else go 
back to step 2. 

5. For each remaining pair (si, S 2 ), check the Frobenius equation for D. 

6. Compute the set of possible values of #J/Fg from the remaining values of 
(si, S 2 ). If there are several values left, go back to step 2. If there is just one, 
return it. 



5.4 Complexity 

We evaluate the cost of this algorithm by counting the number of operations 
in the base field F^. We neglect all the log“ I factors, and denote by M{x) the 
number of field operations required to multiply two polynomials of degree x. 

The first step requires 0{l"^) resultant computations, each of which can be 
done in M{P) operations, and the interpolation of a degree 0{l"^) polynomial 
which can be done in M{1^) operations. For the analysis of the remaining steps, 
we will denote by d the degree of the smallest factor of R{xi) that allows us 
to conclude. We assume moreover that the most costly part of the factorization 
is the distinct degree factorization (which is the case if d is small and if the 
number of factors of degree d is not too large). Then the cost of finding the 
factor is 0{dlog{q))M{l'^). Thereafter the computation relies on manipulations 
of polynomials of degree d and the complexity is 0{l + log{q))M{d), where I 
reflects the I possible values of si and of S 2 and log(g) reflects the Frobenius 
computations. Hence the (heuristic) overall cost for the algorithm is 

+ 0{d\ogq)M{f) + 0{f + \ogq)M{d) 



(16) 
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operations in the base field. 

Now we would like to obtain a complexity for the whole Schoof-like algorithm. 
For that we will keep only the primes I for which d = 0(1); this should occur 
heuristically with a fixed probability (this is an analogue of ‘Elkies primes’ for 
elliptic curves). Then we have to use a set of 0(log(7) primes I, each of them 
satisfying I = 0{logq). Moreover we will assume fast polynomial arithmetic and 
thus M {x) = 0{x) (ignoring logarithmic factors) . Hence the cost of the algorithm 
is heuristically O(log^g) operations in F^. Each operation can be performed in 
0(log^ q) bit operations using classical arithmetic and we get that the complexity 
of the Schoof-like algorithm is 0{log^ q). 



Remark. This analysis is heuristic, but one could obtain a rigorous proof that 
the algorithm runs in polynomial time. The algorithm could also be made de- 
terministic by avoiding polynomial factorizations. However in both cases the 
exponent would be higher than 9. 

6 Lifting the 2-Power Torsion Divisors 

In this section, we will show how to obtain some information on the ^ J /¥q 
modulo small powers of 2. Factoring / gives some information immediately. 
To go further we iterate a method for ‘halving’ divisors in the Jacobian. This 
quickly leads to divisors defined over large extensions, so that the run-time grows 
exponentially. In practice we can use this technique to obtain partial information 
modulo 256, say. 

The divisors of order 1 or 2 are precisely the D = {u{x),0) for which u{x) 
divides f{x) and is of degree at most g. When / has n irreducible factors, then 
it has 2” factors altogether. Exactly half of them have degree at most g, since / 
is square- free of degree 2g + 1. Hence the number of such divisors is 2”“^, and 
2^-1 I Furthermore, when / is irreducible then the 2-part is trivial and 

#J /Wq is odd. 

6.1 Halving in the Jacobian 

Let D = {u{x),v{x)) be a divisor different from O. We would like to find a 
divisor A such that [2]Z\ = D. Note that there are 2^® solutions, any two of 
which differ by a 2-torsion divisor. In general, A is defined over an extension of 
the field of definition of D. 

Writing A = (u(x), v(x)}, we derive a rational expression for the divisor 
[2] A using the formulae of section 1. Then equating this expression with D, we 
get a set of 2g polynomial equations in 2g indeterminates ui and vi with 2g 
parameters Ui and Vi. There are g^ such systems corresponding to the different 
possible weights of D and A. 

We consider the most frequent case where D and A are both of weight g. 
The corresponding system has at most 2^® solutions and these can be obtained 
by constructing a Grobner basis for a lexicographical order, factoring the last 
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polynomial in the basis and propagating the solution to the other polynomials. 
All this can be done in time polynomial in log q provided that the divisor D we 
are dealing with is defined over an extension of bounded degree of . 

In order to speed up the computations in the case where D is defined over a 
large extension, we can avoid repeated Grobner-basis computations and instead 
compute a single generic Grobner basis for the system, where the coefficients 
of D are parameters. As the halving is algebraic over (because the curve is 
defined over F^), the generic basis is also defined over F^. After this computation 
we can halve any divisor D, even when defined over a large extension, by plugging 
its coefficients into the generic basis to get the specialized one. 

We are indebted to Eric Schost who kindly performed the construction of 
this generic Grobner basis for the curves we studied [Sch]. For his construction, 
he made use of the Krone cker package [Lec99] written by Gregoire Lecerf. This 
package behaves very well on these types of problem (lifting from specialized 
systems to generic ones), and it is likely that we would not have been able to do 
this lifting by using classical algorithms for Grobner-basis computations. 



Example. Let C be defined by 

= x^ + 1597 + 1041 x^ + 5503 x^ + 6101 x + 1887 , (17) 

over the finite field Fp with p = 10^^-|-3. We will search for all rational 2-power 
torsion divisors i.e., those defined over Fp. Two irreducible factors of f{x) have 
degree at most 2, they are 

fi = x + 28555025517563816 and f 2 = x + 74658844563359755 , 

Thus there are three rational divisors of order two: Pi = (/i,0), P 2 = (/2j0) 
and Pi + P 2 - The halving method applied to Pi finds four rational divisors of 
order 4. They are (u, v) and (u, —v) where: 

u = x'^ + 1571353025997967 ® -t 12198441063534328 

V = 32227723250469108 x -t 68133247565452990 

and: 

u = x'^ + 70887725815800572 x + 94321182398888258 

V = 42016761890161508 x + 3182371156137467 . 

There are 16 solutions altogether but the others are in extension fields (the 
Grobner bases are too large to include them here!) Applying the method to P 2 
and to Pi + P 2 finds no further rational 4-torsion divisors. By continuing in the 
same manner one finds 8 divisors of order 8, 16 of order 16, 32 of order 32 and 
no more. Thus the 2-part of the rational Jacobian is of the form (Z/2) x (Z/32) 
and hence #J /Fp = 64 mod 128. 

This type of exhaustive search in the base field determines the exact power of 
2 dividing ^ J /Fp. In the next section we show how to find information modulo 
larger powers of 2. 
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6.2 Algorithm for Computing #J /Fg mod 2*^ 

Next we go into extension fields to find some 2^-torsion divisors and we substi- 
tute them into the characteristic equation of the Frobenius endomorphism, to 
determine values of its coefficients modulo 2^ and hence the value of /Fg mod 
2^, for increasing k. 

Algorithm (for g = 2). 

1. Factor / to find a 2~torsion divisor. Halve it to get a 4-torsion divisor D 4 . 

2. Find the pair (si, S 2 ) mod 4 for which xi^i) = Set k to 2. 

3. Compute the generic Grobner basis for halving (weight 2) divisors in the 
given Jacobian. 

4. Build a 2^+^-torsion divisor D 2 k+i by substituting the coefficients of D 2 k in 
the system, computing a root of the eliminating polynomial in an extension 
of minimal degree, and propagating it throughout the system. 

5. For each pair (si, S 2 ) mod compatible with the previously found pair 
modulo 2^, plug D 2 k+i into \ find the pair for which x(JJ 2 '=+i) = 

6. Set k= k+1, and go back to Step 4. 

Note that this is an idealized description of the algorithm. In fact there will 
frequently be several pairs (si, S 2 ) remaining after checking the Frobenius equa- 
tion for one 2 ^-torsion divisor. We can eliminate false candidates by checking 
with other 2^-torsion divisors. It can be costly to eliminate all of them when 
the required divisors are in large extensions; an alternative strategy is to con- 
tinue and expect the false candidates to be eliminated later using 2^+^-torsion 
divisors. 

In this algorithm, we could skip step 3 and compute specific Grobner bases 
at each time in step 4. However, the generic Grobner basis is more efficient and 
allows one to perform one or two extra iterations for the same run-time. 

7 Combining these Algorithms — Practical Results 

We have implemented all these algorithms and tested their performance for real 
computation. Some of them were written in the G programming language, and 
others were implemented in the Magma computer algebra system [BG97]. 

7.1 Prime Field 

In the case where the curve is defined over a prime field Fp, where p is a large 
prime, we use all the methods described in previous sections except for Gartier- 
Manin. We give some data for a ‘random’ curve for which we computed the 
cardinality of the Jacobian. Let the curve C be defined by 

= + 3141592653589793238 -t 4626433832795028841 

-k 9716939937510582097 -t 4944592307816406286 x (18) 

-t 2089986280348253421 , 
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over the prime field of order p = 10^® + 51. The cardinality of its Jacobian is 
#J = 99999999982871020671452277000281660080 , (19) 

and the characteristic polynomial of the Frobenius has coefficients: 

Si = 1712898036 and S 2 = 11452277089352355350 . 

The first step of this computation is to factor f{x). It has 3 irreducible factors, 
thus we already know that = 0 mod 4. 

The second step is to lift the 2-power torsion divisors. The computation of 
the generic halving Grobner basis (done by E. Schost) took about one hour on 
an Alpha workstation. Then we lifted the divisors several times and checked the 
Frobenius equation. In the following table we give the degree of the extension 
where we found a 2^-torsion divisor, and the information on that we got 
(timings on a Pentium 450). 





deg of ext 




deg of ext 


time 


0 mod 2 


1 


16 mod 32 


16 




0 mod 4 


1 


48 mod 64 


32 




0 mod 8 


4 


48 mod 128 


64 


5000 sec 


0 mod 16 


8 


176 mod 256 


128 


9 hours 



The next step is to perform the Schoof-like algorithm. We did so for the 
primes I G {3, 5, 7, 11, 13}. The following table gives the degree of the polynomial 
R{xi) for each I, and the smallest extension where we found an Gtorsion divisor 
(timings on a Pentium 450). 



1 


degree of R{x\) 


degree of ext 




time 


3 


240 


2 


1 mod 3 


1200 sec 


5 


2256 


1 


0 mod 5 


300 sec 


7 


9120 


6 


4 mod 7 


12 hours 


11 


57360 


1 


0 mod 11 


19 hours 


13 


112560 


7 


9 mod 13 


205 hours 



The run-time for ^ = 3 is surprisingly large in this table. For our curve, an 
unlucky event occurs, which becomes rare as I increases. Indeed, after testing the 
Frobenius equation for all the 3-torsion divisors several candidates (si, S 2 ) still 
remain, yielding several possibilities for ^3 mod 3. What this means is that the 
minimal polynomial of (j) is not the characteristic polynomial. Each remaining 
candidate for (si, S 2 ) gives a multiple of the minimal polynomial. By taking their 
GCD we obtain the exact minimal polynomial, from which we can deduce the 
characteristic polynomial^ and #J mod 3. 

In our case, there are 3 pairs left after testing all the 3-torsion points, leading 
to the following candidates for ^3 mod 3. 



(si, S 2 ) mod 3 


mod 3 


X(t) mod 3 


(0,2) 


1 


R + 1 


(1,2) 


2 


R - t+1 


(2,2) 


0 


R + t^ + t+l 



^ See [Kam91] for more about this. 
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The third case is impossible because if #J = 0 mod 3 then we would have 
found a rational 3-torsion divisor earlier. In order to decide between the two 
first cases we determine the minimal polynomial, which is + 1 and thus the 
characteristic polynomial must be + 1)^ and finally = 1 mod 3. 

However to do this we have to build all the 3-torsion divisors. This explains 
why the running time is higher than for I = 5, where we found a rational 5- 
torsion divisor and immediately deduced that = 0 mod 5. 

The final step is the birthday paradox computation. The width of the Hasse- 
Weil interval is roughly 2.5 x 10^®. The search space is reduced by a factor 
2®x3x5x7xllxl3 = 3843840 leaving 6.6 x 10^^ candidates. The search was 
performed on ten Alpha workstations working in parallel and calculated 5 x 10^^ 
operations in the Jacobian. On a single 500 MHz workstation, this computation 
would have taken close to 50 days. 

7.2 Non-prime Fields 

Let C be a genus 2 curve defined over Fpn, where p is a small odd prime. We 
assume that C is not defined over a small subfield, for in that case it is easy to 
compute x(t) using a theorem due to Weil. 

Here the first step is to use Cartier-Manin to get x(t) mod p quickly and then 
continue as before, except that we avoid I = p in the Schoof part. 

Examples: We did not try to build big examples, however we give two medium 
ones. For the first, let the curve C be defined by 

+ x'^ + + tx + 1 , (20) 

over the finite field F330 = F3[t]/(t^° + f — 1). The cardinality of its Jacobian is 

#J = 42391156018493425614913594804 . (21) 

The second example illustrates the advantage given by Cartier-Manin in a 
favorable case where p = 2^® — 15. Let the curve C be defined by 

= x^ + x'^ + x^ + x"^ + X + t , (22) 

over the finite field Fp4 = Fp[f]/(f^ — 17). The Cartier-Manin computation gave 
us #J = 58976 mod p in 17 minutes, and finishing using our other methods gave 

#J = 339659790214687297284652908385855015466 . (23) 

8 Perspectives for Further Research 

The present paper reports on practical algorithms for counting points on hy- 
perelliptic curves over large finite fields and on implementations for genus 2. 
Although it is now possible to deal with almost cryptographic-size Jacobians, 
there is still a substantial amount of work to be done. Some improvements or 
generalizations seem to be accessible in the near future, whereas others are still 
quite vague. Among them we would like to mention: 
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— Extension of the algorithm to even characteristic. This is only a matter 
of translating the formulae, in order to deal with an equation of the form 

+ h{x) y = f{x). The Cartier-Manin part and the lifting of the 2-torsion 
should merge, giving an efficient way to compute the result modulo 2^. For 
the Schoof-like part, the formulae of Cantor’s division polynomials have to 
be adapted, which does not appear to be too difficult. 

— Extension of the Schoof-like algorithm to genus g > 2. The main difficulty 
is that it does not appear possible to avoid manipulation of ideals. 

— More use could certainly be made of the Jacobian of the twist curve. 

— We believe that it may be possible to lift the curve to a local field with residue 
field Fq and use Cartier-Manin to compute x(t) modulo small powers of the 
characteristic. We do not yet know how to compute the lift, however. 

— A major improvement would be to elaborate a genus 2 version of the Elkies- 
Atkin approach for elliptic curves, which would lead to computations with 
polynomials of lower degree. We conjecture that it is possible to work with 
degrees reduced from 0{l"^) to 0{l^). The first task is to construct modular 
equations for Siegel modular forms, instead of classical ones. This requires a 
description of isogenies for each small prime degree, which can be given by 
lists of cosets under left actions of the symplectic group Sp 4 {Z) instead of 
the classical modular group 5^2 (Z). Starting points for studying the relevant 
forms and groups include [Fre83] and [KH90] . This will be explained in more 
detail elsewhere [Har]. 

All the above is the subject of active research. 
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Abstract. A description and an example are given of numerical ex- 
periments which look for a relation between modular forms for certain 
congruence subgroups of SL(3,Z) and Galois representations. 



1 Introduction 

In this paper we review a recently discovered relation between some modular 
forms for congruence subgroups of SL(3, Z) and three dimensional representions 
of Gal(Q/Q) (see [vG-T] and [GKTV]). This relation is the equality of local 
L-factors, for primes p < 173, attached to the modular forms and to the Ga- 
lois representation, see Theorem 4.5. The result gives some evidence for general 
conjectures of Langlands and Glozel [Gl]. 

The first three section follow closely the notes from a seminar talk of the 
first author at the seminaire de theorie des nombres de Paris in January 1995. 
In the first section we briefly recall an instance of the relation between elliptic 
modular forms and Galois representations. In the second section we introduce 
the modular forms for GL(3) and the Galois representations are discussed in 
section three. 

In section four we give some new examples of non-cusp forms for congruence 
subgroups of SL(3, Z) and we describe many of these in terms of classical modular 
forms for congruence subgroups of SL(2, Z). The last section deals with a Hodge 
theoretical aspect of the algebraic varieties (motives in fact) we used to define 
the Galois representations. 

It is a pleasure to thank Avner Ash, Kevin Buzzard, Bas Edixhoven and 
Jasper Scholten, especially for their interest and help concerning Sect. 5.5. 

2 Modular Forms: The GL(2) Case 

Let S 2 {N) be the space of cusp forms of weight two for the congruence subgroup 
ro{N) C SL(2, Z). Let f = J2 G S 2 {N) be a newform, thus oi = 1 and / 
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is an eigenform for the Hecke algebra: Tpf = Qpf for all prime numbers p which 
do not divide N. For such a prime p one defines the local L-factor of / as 

Lpif, s) := (1 - 

note that Lp{f, s) is determined by the eigenvalue Op. 

In case all Op are in Z, / defines an elliptic curve Ef, defined over Q {Ef 
is a subvariety of the Jacobian of the modular curve Xq{N)). The Galois group 
Gal(Q/Q) acts on the ^"-torsion points of this curve which gives an Gadic rep- 
resentation: 

Pf,i '■ Gal(Q/Q) — > GL 2 (Q^). 

The local L-factor of this representation for primes p as above does not depend 
on the choice of the prime i p and is defined by 

Lp{pf, s) := det(/ - pf^i{Ep)p~^)~^ = (1 - trace(p/,^(Fp))p"^ -h p^~‘^^)~\ 

with Ep G Gal(Q/Q) a Frobenius element at p. 

The Eichler-Shimura congruence relation asserts that 

Op = trace(pp^(Fp)) so Lp(f, s) = Lp(pf, s) 

(again with p a prime not dividing N£). Thus we have a method to associate to 
a newform / a (compatible system of Gadic) Galois representation(s) p/^£ such 
that the L-factors agree. This construction has been generalized to newforms 
of any weight (and arbitrary Hecke eigenvalues) by Deligne [D] using Galois 
representations on certain etale cohomology groups of certain Gadic sheaves on 
the modular curve Xo(JV). 

It is a pleasure to observe that recently Wiles proved a partial inverse to the 
construction sketched above: he shows that for a certain class of elliptic curves 
defined over Q the corresponding Galois L-series are the L-series of newforms. 
As is well known, this has been used to prove Fermat’s Last Theorem. 

3 Modular Forms for GL(3) 

3.1 

One can also define modular forms, a Hecke algebra and local L-factors for 
congruence subgroups of SL(3, Z), see below. However, the upper half plane 

H={2GC: 9(2) >0}=^SL(2,R)/SO(2), 

which has a complex structure, is now replaced by SL(3, M)/SO(3) (see [AGG]), 
a real variety of dimension 5 which, for dimension reasons(!), cannot have a 
complex structure. 

In particular, one does not know how to associate algebraic varieties to con- 
gruence subgroups of SL(3, Z) (in contrast to the modular curves in the GL(2)- 
case). Therefore there are no a priori given Galois representations on etale co- 
homology groups which could be related to modular forms for such congruence 
subgroups. 
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3.2 

In the case of SL(2, Z), the space of holomorphic modular forms of weight two 
for a congruence subgroup r' is a subspace of the cohomology group 
This generalizes as follows. 

3.3 

From now on we use the following definition: 

T'o(fV) = G SL(3,Z) I 021 = 0 mod N and 031 = 0 mod fv|. 

The modular forms for Ib(fV) we consider are elements of i?^(/o(fV), C). To 
compute this vector space, we introduce a finite set: 

p2(Z/fV) = {{x,y,z) G {Z/Nf I xZ/N + yZ/N + zZ/N = Z/N} j {ZjNY ■ 

When the elements of this set are viewed as column vectors, there is a natural 
left action of SL(3,Z) on P^(Z/7V). This action is transitive, and the stabilizer 
of (1: 0: 0) equals To(fV). Therefore 

SL(3,Z)/To(fV) 

This relation between lo(fV) and P^(Z/7V) leads to a very concrete descrip- 
tion of the vector space H^{ro{N),C). In fact, its dual i? 3 (r'o(fV), C) can be 
computed as follows: 

3.4 Theorem. 

([AGG], Thm 3.2, Prop 3.12) 

There is a canonical isomorphism between H^roiN), C) and the vector space 
of mappings / : P^(Z/fV) — > C that satisfy 

1 . f{x:y:Y = -/{-W-X'-Yi 

2. f{x:y:z) = f{z:x:y), 

3. f{x: y: z) -h /(-y: x - y: Y + fiv ~ x: -x: y) = 0. 

3.5 

For any a G GL(3, Q) one has a (C-linear) Hecke operator: 

: HYro{N),C) HYro{N),C). 

The adjoint operator T* on the dual space i? 3 (r’o(fV), C) can be explicitly com- 
puted using modular symbols. 

The Hecke algebra T is defined to be the subalgebra of End(iJ^(/o(fV), C)) 
generated by the Tq’s with det(a) relatively prime with N. The Hecke algebra 
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is a commutative algebra and we are interested in eigenforms F G i?^(/b(-/V), C) 
for the Hecke algebra: 

TF=A(T)F, with A:T^C (forallTeT). 

Of particular interest are the Hecke operators Fp = Ta^, which are for a 

(p 0 0\ 

prime p not dividing N defined using Op = 0 1 0 € GL(3, Q). 

Vo 0 ij 

Let Op := \{Ep), for a (given) character A of T and a prime p not dividing 
N, then the local L-factor of a Hecke eigenform F G H^{Fq{N),C) (with the 
additional condition that F is cuspidal) corresponding to A (so EpF = apF) is 

Lp{F, s) = (1 - Opp”^ + Opp^”^® 

where dp is the complex conjugate of Op. The field Kp ■= Q(- ■ ■ , Op, . . .) gener- 
ated by the Hecke eigenvalues of an eigenform F is known to be either totally 
real or is a CM field (a degree 2, non-real extension of a totally real field). 

3.6 

In [GKTV], a list of the Op’s with p < 173 is given for several eigenforms with 
N < 245. Here we list some Op’s of three particularly interesting eigenforms 
(these eigenforms are uniquely determined by their level N and the Op’s listed). 
In case p divides N we write ** for Op. In the three cases listed here Kp = Q(z) 
with = —1. The complex conjugates of the Op’s for a given F are the Hecke 
eigenvalues for another modular form G of the same level. 
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4 Galois Representations 

4.1 

We are interested in relating Hecke eigenforms and Galois representations. In 
particular, given a Hecke eigenform F we would like to find (a compatible system 
of) A-adic Galois representations 

Pf.a : Gal(Q/Q) ^ GL(Wa) 

having the same local L-factors as F. Here A is a prime in a finite extension K\ 
of and W\ is a (finite dimensional) K\ vector space. The local L-factors of 
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Pf,\ (again independent of A) being defined as before (for unramified primes, 
conjecturally those not dividing Nt): 

Lp{pF, s) := det{I - pF,\{Fp)p~^)~^ . 

In particular, we want dimlFA = 3. 

4.2 

The case that Kf is totally real is analyzed by Clozel [C2]. We just recall that if 
in this case such a Galois representation pF,\ exists then pF,\ is selfdual in the 
following sense. 

Consider the Tate-twisted dual Galois representation: 

Pf.a := Vf,\(-2) : Gal(Q/Q) GL(Wa), so p*F,x(Fp) ■= 'PpM^p)- 

Let Qfi, z = 1, 2, 3 be the eigenvalues of pF,i{Fp), then the eigenvalues of p*p^{Fp) 
are (3i := p^fai. Since ^ = Op, = pap (since now dp = ap) and 

Y\oii =P^, the sets of eigenvalues {ai} and {Pi} coincide. 

Thus Lp{pF,s) = Lp{p*p,s) for all p not dividing N and so the (semi- 
simplifications of the) Galois representations are the same. It implies also that 
a subgroup of finite index of the image of Gal(Q/Q) is contained in a group 
G C GL(Wa) with G = PGL(2,iLA)- Examples of this are the Sym^ of Galois 
representations in GL(2,Q^). 

4.3 

We will be especially interested in the non-selfdual case. Since we found several 
examples of Hecke eigenforms F with Kp = Q(z) we will consider that case 
here. To find corresponding Galois representations we use the fact that for any 
algebraic variety X defined over Q, one has a Galois representation on the etale 
cohomology: 

Gal(Q/Q) GL(iLS(XQ,Q,)). 

The point is to find a suitable X and (a subspace of) a suitable iL”j. In case X 
is smooth, projective, and has good reduction mod p, theorems of Grothendieck 
and Deligne imply that the eigenvalue polynomial of Fp acting on Ff2t{X^, Q^) 
has coefficients in Z, is independent of £ and the eigenvalues of Fp have absolute 
value 

The desired equality Lp{F,s) = Lp{pF,s) for the eigenforms F from (3.6) 
(and one expects the same more generally for certain cusp forms, ‘Ramanujan 
conjecture’), implies that the absolute value of the eigenvalues of PF{Fp) must 
be p. Therefore we will consider and take dimX > 1 since dimiL^j = 1 for 
curves. 

A well-known theorem implies that F[‘^^{Xq,Qi) ^ F[‘^^{Sq,Qi) where S 
is a suitable surface contained in X. Thus we restrict ourselves to considering 
Q^) for a surface S. 
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The Galois representation on this Q^-vector space is reducible in general, a 
decomposition is: 

HUS^, Qt) = Tt © NS(%) ©z Q, 

where NS(S'q) is the Neron-Severi group of the surface S over Q (the Galois group 
permutes the classes of divisors modulo a Tate twist) and is the orthogonal 
complement of NS(5 'q) with respect to the intersection form. The intersection 
form is the cup product H1^ x H‘l^ The eigenvalues of Frobenius 

on NS(5 q) © Q are roots of unity multiplied by p, so pf,\, if it exists, should be 
a representation on a subspace of ©q^ K\. 

In case has dimension 3, the Galois representation on it will be selfdual 
(due to the intersection form). To find a 3 dimensional Galois representations 
with traceFp G Z[z] as desired we assume that the surface has an automorphism, 
defined over Q: 

(j ) : S — > S, with <f)^ = ids- 

Thus (f>* : FJgj — > FJgj will commute with the Galois representation. 

Assume moreover that dimT^ = 6 and (jf \ Ti ^ Ti has two 3-dimensional 
eigenspaces W\, W'^ (with eigenvalue ±z): 

T\ := Ti ©Q^ K\ = W\ © W'-y 

with K\ an extension of Qi containing z. Then we have a 3-dimensional Galois 
representation a' on W\. The determinant of a'{Fp) is in general not equal to 
p^ but is x{p)p^ for ^ Dirichlet character y. Twisting a' by this character we get 
a Galois representation 

as. A : Gal(Q/Q) GL(VFa). 

whose L-factors Lp{as, s) are similar to the Lp(F, s) for the eigenforms in the 
example above. 

Note that the intersection form (• , •) restricted to W\ is trivial (it is invari- 
ant under pull-back by 4>* and extends AlA-linearly: (z«i,zc 2 ) = 4 >*W 2 ) 

= {iwi,iw 2 ) = i^{wi,W 2 ) = —{wi,W 2 ) with w\, W 2 G W\). Thus there is no 
obvious reason for as to be selfdual. 

4.4 

Now one has to search for such surfaces. The main problem is that in general 
dimFJgj will be large but rankNS will be small. Thus it is not so easy to get 
dimT^ = 6, see however [vG-T] and [vG-T2] for various examples. 

The most interesting example is given by the one parameter family of surfaces 
Sa which are the smooth, minimal, projective model of the singular, affine surface 
defined in x, y, t-space by 

f = xyix"^ - l){y'^ - l){x'^ - + axy), and {x,y,t)> — > {y,-x,t) 

defines the automorphism <j). In [vG-T], 3. 7-3. 9, we explain how to determine 
eigenvalue polynomials of as,\{Fp), and thus the L-factors, basically using the 
Lefschetz trace formula and counting points on S over finite fields. The main 
result is: 
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4.5 Theorem. 

([vG-T], 3.11; [GKTV], 3.9) The local L-factors of the modular forms for N = 
128, 160, 205 in §3.6 are the same as the local L-factors of the Galois represen- 
tations with a = 2, 1, respectively, for all odd primes p < 173 not 

dividing N. 



4.6 

In [vG-T2] we gave another construction of surfaces S which define 3 dimensional 
Galois representations. These surfaces are degree 4 cyclic base changes of elliptic 
surfaces f . By taking the orthogonal complement to a large algebraic part 
in together with all cohomology coming from the intermediate degree 2 base 
change, one obtains a representation space, similar to Ti, for Gal(Q/Q). Taking 
an eigenspace W\ of the action of the automorphism of order 4 defining the 
cyclic base change finally gives 3 dimensional Galois representations. 

Our main (technical) result is a formula for the traces of Frobenius elements 
on this space in terms of the number of points on £ and S over a finite field 
([vG-T2], Theorem 3.4). This formula allows us to compute the characteristic 
polynomial of Frobenius in many cases. 

We use this result to prove that certain examples yield selfdual representa- 
tions, while others do not. For some of the selfdual cases we can actually exhibit 
2-dimensional Galois representations (defined by elliptic curves) whose symmet- 
ric square seems to coincide with the 3-dimensional Galois representation. 

We did not find new examples of non-selfdual Galois representations with 
the same local L-factors as modular forms, probably because the conductor of 
these Galois representations is too large. We would like to point out that there 
does not seem to be an explicit way to determine the conductor of the Galois 
representation as in terms of the geometry of S (a surface over Spec(Q)). 



5 Non-cusp Forms and Galois Representations 

5.1 

In this section we give an example of the decomposition in Hecke eigenspaces of 
a cohomology group i7^(/b(fV), C). We will take N = 245. This example is also 
mentioned in [GKTV], §3.5 where it is shown that a certain 8 dimensional Hecke 
invariant subspace of i7^(/b(245), C) contains no cusp forms. Here we extend 
this by interpreting most of the 83 dimensional space i7^(/b(245), C) in terms 
of so-called Eisenstein liftings of classical elliptic cusp forms and of Eisenstein 
series. 

As before, if F e il^(Fo(245), C) is an eigenform for all Hecke operators, we 
denote hy Kp the field generated by all eigenvalues of the Hecke operators on 
F. As a first step towards the decomposition we have the following Proposition. 
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Proposition 1. The cohomology group i?^(/o(245), C) decomposes as 
i?3(Po(245), C) = Fi © P 2 © V '3 © P 4 © ^5 
(as a module over the Hecke algebra), with 

— dimyi = 25 and Vi is generated by eigenforms F with Kp = Q; 

— dimV 2 = 16 and V 2 is generated by eigenforms F with Kp = Q(-\/2); 

— dimVa = 16 and V 3 is generated by eigenforms F with Kp = Q(vT7); 

— dim 14 = 8 and V 4 is generated by eigenforms F with Kp = Q(-\/2, 3); 

— dimVs = 18 and V 5 is generated by eigenforms F with Kp = Q(-y/— 3). 

None of the spaces Vi, . . . ,¥5 contains a non-zero cuspform; in fact, these spaces 
are generated by Eisenstein liftings or (in the case of V 4 and V 5 ) twists of such 
by cubic Dirichlet characters. 

5.2 

With notations as given in [GKTV] §3.5, one has V 4 = 14 © 14, hence this case 
of the above proposition is already described in loc. sit. 

We briefly recall the two types of Eisenstein liftings of classical modular forms 
here. Let / be a normalized elliptic cuspform of level N and weight 2, which is an 
eigenform for the Hecke operators T„ with (n, N) = 1. Also, we allow / to be the 
normalized Eisenstein series of weight 2: / = so Op = 

p+1, compare e.g. [Ko] for notations. The Fourier coefficients in the g-expension 
/ = qF F a-iC^ F ■ ■ ■ define a Dirichlet series L{f, s) = This series 

has an Euler product expansion with Euler factors (1 — QpP~^ F p^~‘^^)~^ for 
primes p which do not divide N (in case / is the Eisenstein series, these factors 
are (1 — p“^)“^(l — 

Given /, one constructs two eigenclasses Fi, F 2 G F[^{Fo{N),C). The Fi has 
eigenvalue pOp F 1 for the pth Hecke operator Ep, and F 2 eigenvalue UpF p^. 

On the Galois side of the Langlands correspondence, it is relatively easy 
to describe these liftings. Namely, if / corresponds to a 2 dimensional A-adic 
representation space V for Gal(Q/Q), then Fi corresponds to E(— 1) © Qa( 0) 
and F 2 to E © Qa(— 2) where is the 1 dimensional A-adic representation 

space on which the Galois group acts by the — n-th power of the cyclotomic 
character (thus Fp acts as p“”). In case / is the Eisenstein series, we have 
y = Qa(0) © Qa(— 1 ) and the two lifted representations coincide (both are 
Qa(0)©Qa(-1)©Qa(-2)). 



5.3 

There exists a unique normalized cuspform of weight 2 and level 35 which has 
Q-rational Fourier coefficients. This form yields 2 eigenclasses in H^{Fo{35), C); 
from the theory of oldforms [Ree], each of these appears three times at level 
35 • 7 = 245. 
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Similarly, the modular form corresponding to the CM elliptic curve of con- 
ductor 49 gives rise to six oldforms which are Eisenstein liftings. 

Starting from the Eisenstein series, one finds 7 forms at level 245 all with 
eigenvalues 1 + p + p^. 

Finally, from tables of Cremona (as well as from unpublished tables of Cohen, 
Skoruppa and Zagier) it follows that there exist 3 (elliptic) newforms of level 245 
which are Hecke eigenforms with rational eigenvalues. Each of them gives us two 
Eisenstein liftings. 

Adding up, we now have 6-|-6-|-7-|-6 = 25 eigenclasses of level 245 with ratio- 
nal eigenvalues. Our calculations made for the tables in [GKTV] revealed that, 
e.g., the Hecke operator E 2 has precisely 25 rational eigenvalues (counted with 
multiplicity). Hence the conclusion is, that the space V\ given in Proposition 1 
indeed has dim V\ = 25, and it is generated by Eisenstein liftings as claimed. 



5.4 

The cases 02,^3 are completely analogous. For V2, we note that there exist 
newforms of weight 2 and level 245 with g-expansion q + V2q^ + (1 + + ■ ■ ■ 

and q+{l + \/2)q‘^ + {l — \/2)q^ + . . . respectively. These together with their Galois 
conjugate forms and their twists by the quadratic Dirichlet character modulo 7 
give us 8 newforms of level 245. Each of them yields two Eisenstein liftings, and 
this precisely describes the space V2 of dimension 16. 

Similarly, there are exactly two (conjugate) newforms of level 35 with Fourier 
coefficients generating Q(-\/r7). They provide 2-2 = 4 Eisenstein liftings of level 
35, and hence 3 • 4 = 12 oldforms of level 245. Twisting the newforms by the 
quadratic character modulo 7 yields newforms of level 245, and from these we 
find another 4 Eisenstein liftings. In this way, V3 is generated. 



5.5 



Having described Vi, . . . , V4 (the latter space was already treated in [GKTV]), 
and observing from Table 3.3 that dimi7^(/o(245), C) = 83, we conclude we still 
have to describe a Hecke-invariant space of dimension 83 — (25-I-16-I-16-I-8) = 18. 
To this end, we mention that at level 49 = 245/5, our programs found a 6 
dimensional Hecke invariant subspace on which the operator E 2 acts with 6 
(pairwise conjugate, pairwise different) eigenvalues in Q(-\/^^. Hence this space 
yields eigenforms with Kp = Q(-y/— 3). Moreover, it lifts to a Hecke invariant 
subspace of dimension 3 • 6 = 18 at level 245, which therefore exactly equals the 
summand V5 of EI^ we did not describe yet. 

As an example, the eigenvalues of the operator E 3 on V5 are 03,^,030;, 
03W, asoJ and (I 3 UJ where + oj + 1 = 0 and 03 = — 5 — 3-\/— 3. This situation is 
explained as follows. The Euler factor that corresponds to a Hecke eigenclass is 
obtained using the polynomial — UpX'^+pbpX—p^ , where Op is the eigenvalue 
of the operator Ep. The number bp similarly corresponds to the operator Dp = 
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o\ 

0 G GL(3,Q). If the eigenclass is cuspidal, 
1/ 

then bp is the complex conjugate of Op. This in fact follows from the fact that the 
associated automorphic representation is unitary in that case. In our situation 
however, a computation shows that 63 = 03 yf 03. Hence the representation 
cannot be unitary and therefore the eigenclasses here are not cuspidal. 

Based on calculations for primes < 131, the Hecke eigenvalues seem to be 
as follows. For p yf 5, 7 we have bp = Up = x(p)('0(p) + P + with y, ■(/; 

Dirichlet characters modulo 7 of order dividing 3. This corresponds to the sum 
of 1-dimensional Galois representations 

ixi’ G Qa(0)) © (x G Qa(- 1)) © (xV’^ © Qa(-2)). 

6 Variations of Hodge Structures of Weight Two 
6.1 

In all our constructions for Galois representations we consider a subspace Ti C 
This subspace is defined using algebraic cycles, thus there exists 
also a Betti realization Tz C i7^(S'(C), Z) (of the motive T) which is a polarized 
Hodge structure of weight two. We recall the relevant definitions and the main 
results of Griffiths and Garlson on the moduli of the Tz’s. 

The main point is the essential difference with the weight one case (which 
is essentially the theory of abelian varieties). In the weight one case, one has a 
universal family of abelian varieties over suitable quotients of the Siegel space. 
In the weight two (and higher) case, the analogy of the Siegel space is a certain 
(subset of a) period domain, but in general (and in particular this is the case 
with the Tz under consideration), the (polarized) Hodge structures obtained 
from algebraic varieties do not fill up the period space. In fact we will see that 
the Hodge structures like Tz are parametrized by a 4-dimensional space, but 
those that come from geometry have at most a 2-dimensional deformation space 
(and imposing an automorphism of order 4 as we do implies a 1-dimensional 
deformation space). 

It is not clear whether these period spaces (or the subvarieties parametrizing 
‘geometrical’ Hodge structures) have good arithmetical properties like Shimura 
varieties. 

6.2 

Recall that a Z-Hodge structure V of weight n is a free Z-module of finite rank 
together with decomposition: 

Rc := R ©z c = ©p+9=„yp’«, with 

where the are complex vector spaces and the bar indicates complex conju- 
gation (given by v® z = v ®~z). 



(p 0 

defined using /Jp := 0 p 

\0 0 
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A rational Hodge structure Vq is a finite dimensional Q- vector space with a 
similar decomposition of Vc := Vq®qC. Thus a Z-Hodge structure V determines 
a rational Hodge structure on Vq := H Q. 

A (rational) Hodge structure Vq determines an M-linear map, the Weil oper- 
ator: 

J : Vr := Vq G)q M — > Hr with JcVp^g = i^~'^Vp^g 

for all Vp^g € and Jq is the C-linear extension of J. One has = (— 1)” 
since = (— = (— 1 )p+' 3. Thus J determines a complex structure on 

Vr in case V has odd weight. 

A polarization on a rational Hodge structure Vq of weight n is a bilinear map 

S' : Vq X Vq — > Q, 'Pc{vp,q, Vr,s) = 0 unless p+r = q+ s = n 

(intrinsically: S' : Vq O Vq ^ is a morphism of Hodge structures) which 

satisfies the Riemann relations, that is, for all v, w G Vjr: 

Jw) = Jv), <f'(u, Ju)>0 (if u yf 0) 

thus S' defines an inner product J-) on Vr. 

One easily verifies, using the first property, that 'P{Jv, Jw) = 'P{v,w), since 
also Jw) = <f'(w, J^u) = (— l)"<f'(w, v), a polarization is symmetric if n is 

even and antisymmetric if n is odd. 



6.3 

For a smooth complex projective variety X the cohomology groups iJ”(A, Q) 
are polarized rational Hodge structures of weight n. One writes HP’’^{X) := 
i?”(A, C)P’^. In case A is a surface, the cup product on H‘^{X,Q) (note that 
H‘^{X,Q) = Q) gives (—1 times) a polarization on the primitive cohomology 
Hprim- particular it induces a polarization S' on the sub-Hodge structure 
Tq = NS"'' of i?^(S'(C), Q) which we consider. 



6.4 



Let Tz be a Hodge structure of weight 2 and rank 6 with 



Tq = dimTP’« = 2 



for all p, q. Then one easily verifies that: 



Tr = VFi © VV 2 with 



r VFi :=TRnri’i 

\ W2 := Tr n (T2.0 © ro-2) 



For V GWi C we have Jv = v and thus >F(u, u) = !F(u, Jv) > 0, so is 
positive definite on VFi . Hence we can choose an M basis /i , /2 of VFi which is 
orthonormal w.r.t. and which is a C-basis of = Wi ©r C. 
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For V G W 2 we have v = W 2 ,o + ^^o ,2 thus Jv = —v and so is negative definite 
on W 2 - Let :=ei+ei, U 2 := 62 +62 be an orthonormal basis for (—1/2)!F on 
W 2 with d, 62 G Then ei, 62 is a C-basis of (and thus ei, 62 is a C- 
basis of T°’^). Note —2 = !F(ei + ei, ei + ei) = <F(ei, ei) + <F(ei, ei) = 2tf'(ei, ei) 
(since is symmetric). In this way one finds >F(efc, e/) = —5ki (Kronecker’s delta) 
thus <Fc is given by the matrix Q on the basis ei, 62 , /i, / 2 , ei, 62 of Tc\ 

) 



Q = 



0 I 
-/o 



6.5 

We consider first order deformations of the polarized Hodge structure Tz as in 
§6.4. Thus we fix the Z-module and the bilinear map and consider deformations 
of the Hodge structure induced by deformations of an algebraic variety X with 
Tz C H‘^{X, Z), that is, of the direct sum decomposition Tc = 0TP’^. 

The first order deformations of a smooth complex projective algebraic variety 
X are parametrized by H^{X, Ox) with Ox the tangent bundle of X (Kodaira- 
Spencer theory). The isomorphisms HP’’^{X) = H'^{X, Qp) and the contraction 
map Ox ®Ox ^x ^ product map: 

H\X,Ox)®HP’\X) FfP-L9+i(X). 

Thus, for any n, we obtain a map, called the infinitesimal period map: 

(5 : H\X,Ox) ©p+,=„Hom(i7P'«(X),i7P-L9+i(X)). 

Griffiths proved that for 6 G H^{X, Ox), the deformation of the Hodge structure 
induced by the deformation of X in the direction of 9 is essentially given by S{9). 

The subspace Q(S) of (Bp+q=nHom(ffP’‘^(X) , (X)) satisfies (at least) 

two conditions. The first comes from the polarization (see §6.6), the second is an 
integrability condition found by Griffiths which is non-trivial only if the weight 
of the Hodge structure is greater than one (see §6.8). 

We will now spell out the restriction of these conditions to the sub Hodge 
structure (Bp+q=nHom(TP’‘^ , TP~^’‘^~^^) . 

6.6 

The condition that ^ G ©p+q=„Hom(rP’^, TP“^’'3+^) C End(Tc) preserves the 
polarization on T, is that >F((/ + tip)v, (/ + tip)w) = <F(ti, w) when = 0: 

'Fci'>P{v),w) + <I'c{v,^{w)) = 0 Wx,yeTc 

This condition implies that if ip preserves S', then it is determined by ip 2 where 

(^2,V’i) GHom(r"’°,rLi)0Hom(rLi,T°’"). 

In fact, for all v G and w G we have: <Fc(u, •0i(w)) = —'Pci'^ 2 {v),w). 
Since <?c identifies (T‘^. 2 j<i-[ta/ this equality thus defines 4>i{ w) in terms 

of (p 2 - 
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6.7 

With respect to the basis of Tc considered in 6.4, G Hom(r^’°, 0 

Hom(T^’^, C End(Tc) is given by a matrix N and the condition on 
becomes *'NQ + QN = 0 so: 

/ 0 0 0 \ 

TV = ^ 0 0 and B = *A 

\Q Bo) 

where the matrix A (defining (j)2 '■ can be chosen arbitrarily. This 

gives an isomorphism between the space M^iC) of 2 x 2 complex matrices and 
polarization preserving deformations ip: 

/O 0 0\ 

M2(C) ^ (Hom(r2’°,ri’i)0Hom(ri’\r°’2))^, xli — > N{A) := xl 0 0 . 

\ 0 *AoJ 

Thus we have a four dimensional deformation space. In case of Hodge structures 
of weight one, preserving the polarization is the only infinitesimal condition. 
Here, in the weight two case, there is however another condition. 

6.8 

An important restriction, discovered by Griffiths, on the image of S is: 

[Im(5, Im(5] = 0 i.e. S(a) o 5 {( 3 ) = 5 {( 3 ) o <5(a), 

for all a, P G H^{X, Ox), so 9(<5) is an abelian subspace of End(Tc)- For Hodge 
structures of weight n > 2 this imposes non-trivial conditions on the (dimension 
of) the image of S. We consider again our example (cf. [Ca]). 

6.9 

We already determined the polarization preserving deformations in §6.7. Using 
the same notation we find that Griffiths’ condition is: 

N{A)N{B) = N{B)N{A) thus *AB = *BA. 

This condition can be rephrased as saying that ^AB must be symmetric. 

Thus the image of S is at most two dimensional and if it is two dimensional 
with basis X(A), N{B) then A and B span a maximal isotropic subspace of the 
symplectic form: 

E : M 2 {C) X iW2(C) — > C, E{A, B) := 011612 — 012611 0 021622 ~ 022621 = 0. 

We recall that we also have an automorphism (p* : T ^ T, preserving this 
automorphism gives another non-trivial condition on the deformations. Thus 
the one parameter in our surfaces Sa (and in the other examples from [vG-T2]) 
is the maximal possible. 
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Modular Symbols and Hecke Operators 
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Abstract. We survey techniques to compute the action of the Hecke 
operators on the cohomology of arithmetic groups. These techniques can 
be seen as generalizations in different directions of the classical mod- 
ular symbol algorithm, due to Manin and Ash-Rudolph. Most of the 
work is contained in papers of the author and the author with Mark 
McConnell. Some results are unpublished work of Mark McConnell and 
Robert MacPherson. 



1 Introduction 



1.1 

Let G be a semisimple algebraic group defined over Q, and let E C G(Q) be an 
arithmetic subgroup. The cohomology of F plays an important role in number 
theory, through its connection with automorphic forms and representations of the 
absolute Galois group Gal(Q/Q). This relationship is revealed in part through 
the action of the Hecke operators on the complex cohomology H*{F; C). These 
are endomorphisms induced from a family of correspondences associated to the 
pair (E, G(Q)); the arithmetic nature of the cohomology is contained in the 
eigenvalues of these linear maps. 

For E C SL„(Z), the modular symbols and modular symbol algorithm of 
Manin [17] and Ash-Rudolph [8] provide a concrete method to compute the Hecke 
eigenvalues in H''{F ; C), where v = n{n -b l)/2 — 1 is the top degree (§2). These 
symbols have allowed many researchers to fruitfully explore the number-theoretic 
significance of this cohomology group, especially for n = 2 and 3 [3,7,5,21,22]. 
For all their power, though, modular symbols have limitations: 

— The group G must be the linear group SL„. 

— The cohomology must be in the top degree v. 

— The group E must be a subgroup of SL„(Z), or more generally SL„(E), 
where E is a Euclidean ring of integers of a number field. 



1.2 

In this article we discuss new techniques to compute the Hecke action on the 
cohomology of arithmetic groups that can be seen as generalizing the modular 
symbol algorithm by relaxing the three restrictions above. First in §3 we relax 
the first restriction by replacing the linear group SL„ with the symplectic group 
Sp 2 „ [14]. Next in §4, we relax the second restriction and consider computations 
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in H''~^{r), where F C SL„(Z) and n < 4 [13]. Finally, in the last two sections 
we relax all three restrictions, and consider arithmetic groups associated to self- 
adjoint homogeneous cones (§5) [12,15], and arithmetic groups for which a well- 
rounded retract is defined (§6) [16]. The first class includes SL„(Oj^), where Ok 
is the maximal order of a totally real or CM field, as well as arithmetic groups 
associated to the positive-definite 3x3 Hermitian octavic matrices. The second 
class includes arithmetic subgroups of SL„(H), where H is a division algebra 
over Q. 

Most of this work is contained in papers of the author [14,12,13] or the 
author in joint work with Mark McConnell [15]. The last section is a summary 
of unpublished results of Robert MacPherson and Mark McConnell [16]. We have 
omitted other work, notably that of Bygott [10], Teitelbaum [20], and Merel [18], 
because of lack of space and/or author’s expertise. It is a pleasure to thank Avner 
Ash, Robert MacPherson, and Mark McConnell for many conversations about 
these topics. 

2 Classical Modular Symbols 
2.1 

We begin by recalling the classical modular symbol algorithm following Ash- 
Rudolph [8]. For simplicity we consider subgroups of SL„(Z), although every- 
thing we say can be generalized to subgroups of SL„(R), where i? is a Euclidean 
maximal order in a number field (cf. [11]). 

Let r C SL„(Z) be a torsion- free finite-index subgroup, and let m € M„(Q), 
the n X n matrices over Q. We want to show how to use m to construct a class 
in F[''{r). To this end, let X be the symmetric space SL„(M)/SO(n), let X 
be the bordification constructed by Borel-Serre [9], and let dX = X \ X. Let 
M = r\X, M = r\X, and dM = M \ M . Then M is a smooth manifold with 
corners, and H*{F) = H*{M). We have an exact sequence 

Hn-i{dX) ^ il„(A, dX) ^ H„{M, dM) ^ H^M) (1) 

coming from the sequence of the pair {dX, A), the canonical projection X M, 
and Lefschetz duality. Moreover, the boundary dX has the homotopy type of 
the Tits building B = Bsl associated to SL„(Q). This is an {n — l)-dimensional 
simplicial complex whose fc-simplices A are in bijection with flags F of rational 
subspaces 

A = {0 C Fi C . . . C Ffc+i C Q"}; 

we have Z\ C A' if and only ii F C F'. 

Any ordered tuple of nonzero rational vectors determines a maximal rational 
fiag by defining Fk to be the span of the first k vectors. Hence if m G M„(Q) has 
nonzero columns, the different orderings of the columns determine n\ different 
oriented {n— l)-simplices in B. These simplices can be thought of as an oriented 
simplicial cycle giving a class [m] G Ffn-i{B) = Ffn-i{dX). The class [m] is 
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called a modular symbol, and these classes span Hn-\{B). According to Ash- 
Rudolph, the map Hn-\{B) — > H''{r) induced by (1) is surjective; hence the 

(duals of) the modular symbols span H''{r). 

2.2 

Write [m] = [mi, . . . , m„], where each column m^ G Q” \ {0}, and let be 
the Z-module generated by the classes of the symbols [m] . Using the description 
in §2.1, one can show that elements of satisfy the following relations: 

1. [gmi, m 2 , . . . , m„] = [m], for g G 

2. [mo-(i), ■ ■ ■ , mo-(n)] = sgn((T)[m], for any permutation a. 

3. [m] = 0 if det m = 0. 

4. X^r=o(~^)*[’^ 0 ) ■ ■ ■ ) iTii, ■ ■ ■ , m„] = 0, for any n + 1 vectors mo, . . . , m„ (the 
“cocycle relation”). 

By the first relation, is generated by those [m] such that rm is integral 
and primitive for all z. If m G SL„(Z), then [m] is called a unimodular symbol. 
We have the following fundamental result of Manin (rz = 2) and Ash-Rudolph 
(n > 2): 

Theorem 1. [17,8] Any modular symbol is homologous to a finite sum of uni- 
modular symbols. 

We sketch the proof. If jdetmj > 1, then one can show there exists v G 
Z” \ {0} such that 

0 < I det mi(u)| < I det m|, for z = 1, . . . , n. (2) 

where mi(v) is the matrix obtained by replacing the column m^ with v. Such a v 
is called a reducing point for m. Then applying the cocycle relation to the tuple 
z;, mi, . . .,m„ yields an expression for [m] in terms of the symbols [mi(z;)]. By 
induction this completes the proof. 

This process of rewriting a modular symbol as a sum of unimodular symbols 
is called the modular symbol algorithm. Using this algorithm one can compute 
the action of the Hecke operators on as follows. There are only finitely 

many unimodular symbols mod F , and from them one can select a subset dual 
to a basis of H''{F). A Hecke operator acts on the modular symbols by taking a 
unimodular symbol into a sum of nonunimodular symbols. Hence the modular 
symbol algorithm allows one to compute the Hecke action on a basis, from which 
one can easily compute the eigenvalues. 

3 Symplectic Modular Symbols 

3.1 

For the first generalization we replace the linear group with the symplectic group 
[14]. Let U be a 2zz-dimensional Q- vector space with basis {ei, . . . , e„, en, ■ ■ ■ , ei}, 
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where i := 2n + 1 — i. Let ( , ): y x y — > Q be the nondegenerate, alternating 
bilinear form defined by 

{ 1 if j = z with i < j 
-1 ifj = zwithz>j 
0 otherwise. 

The form ( , ) is called a symplectic form, and the symplectic group Sp 2 „(Q) is 
defined to be the subgroup of SL 2 „(Q) preserving ( , ). 

3.2 

Much of §2 carries over without change, but there are some new wrinkles coming 
from the geometry of the symplectic form. Recall that an isotropic subspace is 
one on which the symplectic form vanishes, and that maximal (necessarily n- 
dimensional) isotropic subspaces are called Lagrangian. Then the symplectic 
building ,8sp has a fc-simplex for every length (fc+ 1) fiag of isotropic subspaces. 
Since the columns of a symplectic matrix m satisfy 

{rrii, rrij) = 0 if and only if z yf j, (3) 

it is easy to see that m determines 2"-rz! oriented simplices of maximal dimension 
in ,8sp. 

Furthermore, the arrangement of these simplices in ,8sp differs from the linear 
case. Suppose we use the columns of m to induce points in the projective space 
Then the Lagrangian subspaces spanned by the columns of m become 
(zz— l)-dimensional fiats arranged in the configuration of a hyperoctahedronf This 
time m determines a class [m] G Hn-i(Bsp), and as m ranges over all rational 
matrices with columns satisfying (3), the duals of the classes [m] span H''{r). 

3.3 

As a first step towards a symplectic modular symbol algorithm, one must un- 
derstand the analogues of the relations from §2.2. The analogues of 1-3 are only 
slightly different to reflect the hyperoctahedral symmetry. The cocycle relation, 
however, is more interesting. A symbol [m] and a generic nonzero rational point 
V GV determine 2n modular symbols [m,i{v)\ as follows. For any pair (z, j) with 
z yf j, we define points by 

WLij := {v, mj)mi — {v, mi)mj. 

Let [mi{v)] be the modular symbol obtained by replacing m, with v, and replac- 
ing the nij with j ^ {z,z} by rriij. Then one can show [m] = '^ei[mi{v)] for 
appropriate signs £j. 

For an example of this relation, consider Figure 1. The figure on the left 
shows the cocycle relation for Sp 4 in terms of a configuration in The black 
dots are the points corresponding to the rrii, the grey dot correspond to v, and 
the triangles to the points rriij. 

^ Recall that a hyperoctahedron is the convex hull of the 2n points {±e | e € E}, 
where E is the standard basis of 
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3.4 

Now we can describe the symplectic modular symbol algorithm. Let m € M 2 „(Z) 
have columns satisfying (3). Then det m = nr=i show that 

if I det m| > 1, there exists a vector u G Z” \ {0} such that 

0 < I (mi , u) I < (mi , m,) , for z = 1 , . . . , 2n. 

We can apply v to [m] in the cocycle relation alluded to in §3.3, but we will 
unfortunately find that | det mi{v)\ > \ det m| in general. However, all is not lost. 
It turns out that for fixed z and fixed v, the 2n — 2 vectors {niij \ j ^ z,z} 
form a tuple that can be regarded as a symplectic modular symbol associated to 
Sp 2 „_ 2 - By induction one knows how to make these symbols unimodular, and 
this allows one to further reduce the [mi(u)] (cf. the right of Figure 1). 




Fig. 1. G = Sp 4 . On the left, the outer square is the original symbol [m], and 
the four smaller squares are the symbols [m,i{v)\. On the right, each modular 
symbol has been further reduced by applying the modular symbol algorithm to 
Sp 2 = SL 2 modular symbols. 



4 Below the Cohomological Dimension 



4.1 

We return to the case of SL„. As said before, a limitation of the modular symbol 
algorithm is that one can compute the Hecke action only on the top degree 
cohomology. For zz < 3 this cohomology group is very interesting: it contains 
cuspidal classes, i.e. classes associated to cuspidal automorphic forms. If n > 4, 
however, the top degree cohomology group no longer contains cuspidal classes. 
In particular, if zz = 4, one is really interested in computing the Hecke action on 
H^{r). For instance, recent work Jasper Scholten has constructed 4-dimensional 
representations of Gal(Q/Q) that should be related to a cuspidal Hecke eigenform 
in H^{r) for some F C SL 4 (Z) [19]. The modular symbol algorithm, however, 
applies only to Ff^{F). 

In this section we describe an algorithm that for zz < 4 allows computation 
of the Hecke action on H''~^{F) [13]. However, there is one caveat: we cannot 
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prove the algorithm will terminate. In practice, happily, the algorithm has always 
converged, and has permitted investigation of this cohomology [4]. 

4.2 

To compute with lower degree cohomology groups, we use the sharbly complex 
S'* [2]. For fc > 0, let Sk be the ZT-module generated by the symbols u = 
[v\, . . . ,Vn+k]i where Vi G Q \ {0}, modulo the analogues of relations 1-3 in 
§2.2. Elements of Sk are called k-sharblies. Let d: Sk Sk-i be the map u h—> 
. . . , Fi, . . . , r^n+fc], linearly extended to all of Sk- There is a map 
So ^ Mn giving a ZT-free resolution of A4„, and one can show that this implies 
iL"-'=(T;C)^iLfc(S*G)C). 

As in §2.2, it suffices to consider fc-sharblies u = [ui, . . .,Vn+k] with all Vi 
integral and primitive. Any modular symbol of the form . . . , where 
{zi, . . . , in} C {1, . . . , n + k}, is called a submodular symbol of u. 

Let ^ = X) ?T-(u)u be a sharbly chain. We denote by ||^|| the maximum ab- 
solute value of the determinant of any submodular symbol of The chain ^ is 
called reduced if ||^|| = 1. It is known that reduced 1-sharbly cycles provide a 
finite spanning set of ; C) for n < 4. 

Since the Hecke operators take reduced sharbly cycles to nonreduced cycles, 
our goal is to apply the modular symbol algorithm simultaneously over a nonre- 
duced 1-sharbly cycle ^ to lower the determinants of the submodular symbols. 
Hence we are faced with two problems: first, how do we combine reducing points 
with the original 1-sharbly ^ to produce a new 1-sharbly homologous to 
second, how do we choose the reducing points so that ||^'|| < ||^||? 

4.3 

To address the first issue we do the following. Suppose u = [ui, . . .,u„+i] sat- 
isfies n(u) yf 0, and for i = l,...,n-|- 1, let be the submodular symbol 
[ui, . . . , Ui, . . . , u„+i]. Assume that all these submodular symbols are nonuni- 
modular, and for each i let Wi be a reducing point for v^. 

For any subset I C {1, . . . , n -I- 1}, let uj be the 1-sharbly [ui, . . . , Un+i], 
where Ui = Wi if i G I, and Ui = Vi otherwise. Then we have a relation in Si 
given by 

u=-^(-l)#V. (4) 

1^0 

Geometrically this relation can be expressed using the combinatorics of the hype- 
roctahedron [13, §4.4]. More generally, if some happen to be unimodular, then 
one can construct a similar relation using an iterated cone on a hyperoctahedron. 

4.4 

Now we apply the construction in §4.3 to all the 1-sharblies u with n(u) yf 
0, and we choose reducing points T-equivariantly. Specifically, if v and v' are 
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two submodular symbols of ^ with yv = v', then we choose the corresponding 
reducing points such that yw = w' . After applying (4) to all the u we determine 
a new 1-sharbly cycle Clearly is homologous to We claim that ||^'|| should 
be less than ||^||. 

To see why this should be true, consider the 1-sharblies uj on the right of (4) . 
Of these 1-sharblies, those with = 1 contain the among their submodular 
symbols. We claim that since ^ is a cycle mod F, and since the reducing points 
were chosen T-equivariantly over these 1-sharblies will not appear in . Hence 
by construction we have eliminated some of the “bad” submodular symbols from 

e 

4.5 

Unfortunately, this doesn’t guarantee that ||^'|| < ||^||. The problem is that we 
have no way of knowing that the submodular symbols of the uj with > 1 
don’t have large determinants. Indeed, this brings us back to the second question 
raised in §4.2, since if the reducing points are chosen naively, these submodular 
symbols will have large determinants. However, we claim that one can (con- 
jecturally) choose the reducing points “uniformly” over ^ in a sense by using 
LLL-reduction, and that this problem doesn’t occur in practice. In fact, in thou- 
sands of computer tests and in applications, we have always found ||^'|| < ||^|j if 
n < 4 and ||^|| > 1. We refer the interested reader to [13] for details. 

5 Self-Adjoint Homogeneous Cones 

5.1 

Now we describe a different approach to computing the Hecke action that can 
be found in [12,15]. The main idea is to replace modular symbols and sharbly 
chains with chains built from rational polyhedral cones, and to replace “uni- 
modularization” with moving the support of a chain into a certain canonically 
defined set of rational polyhedral cones. The results of this section apply to any 
arithmetic group that is associated to a self-adjoint homogeneous cone; the re- 
duction theory in this generality is due to Ash [6, Ch. 2]. However, for simplicity 
we describe the results in the context of Voronol’s work reduction theory of real 
positive-definite quadratic forms [23] . 

Let V be the real vector space of all real symmetric n x n matrices, and 
let C be the subset of positive-definite matrices. Then C is a cone, i.e. C is a 
convex set closed under homotheties and containing no straight line. The group 
SL„(Z) acts on V preserving C, and the action commutes with homotheties. In 
fact, modulo homotheties C is isomorphic to A = SL„(M)/SO(n); this exhibits 
a hidden linear structure of the symmetric space X. 

Let C be the closure of C in U. Voronoi showed how to construct a set V of 
rational polyhedral cones in C such that 
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1. r acts on V. 

2. If (T G V then so is any face of a. 

3. If (T, r G V, then <t n r is a face of each. 

4. Modulo r, the set V is finite. 

5. The intersections a \^C cover C . 

The cones V provide a reduction theory for C in the following sense: any x G C 
lies in a unique cone (j{x) G V, and the number of 7 G T such that 'j-a{x) = cr(x) 
is bounded. Given x G C, there is an explicit algorithm, the Voronot reduction 
algorithm, to find a{x). 

The Voronoi cones descend modulo homotheties to induce a decomposition 
of X into cells. Furthermore, we can enlarge C to ^cone C such that, if X 
denotes C modulo homotheties, then the quotient F\X is compact. This Satake 
compactification of F\X is singular in general, but nevertheless can still be 
used to compute F[*{F ; C). For us, the salient points are that the images of the 
Voronoi cones induce a decomposition of C, with all the properties listed above, 
and that the Voronoi reduction algorithm extends to the boundary dC := C\C. 



5.2 

Now let be the complex over C generated by all simplicial rational polyhedral 
cones in C, and let CY be the subcomplex generated by Voronoi cones. ^ For any 
chain ^ G Cf , let supp ^ be the set of cones supporting The complex Cf is 
analogous to the sharbly complex, and the subcomplex CY to the subcomplex 
generated by the reduced sharblies. In general, however, CY is not isomorphic 
to the complex of reduced sharblies. Cycles ^ G CY can be used to compute 
F[*{F), but the image T(^) of ^ under a Hecke operator won’t be supported on 
Voronoi cones. Hence we must show how to push T(^) back into CY ■ 

To accomplish this we have essentially two tools — we can subdivide the cones 
in supp T(^), and we can use the Voronoi reduction algorithm to determine the 
con^any point lies in. We apply these as follows. Using the linear structure 
on C, we first subdivide T(^) very finely into a chain . Then to each 1-cone 
T G supp we assign a 1-cone pr G dC, and we use the combinatorics of to 
assemble the pr into a cycle homologous to We claim that if is constructed 
so that 1-cones r G supp lie in the same or adjacent Voronoi cones, then the 
Pr can be chosen to ensure G C^. 



5.3 

We illustrate this process for SL 2 ; more details can be found in [12]. Modulo 
homotheties the three-dimensional cone C becomes the extended upper halfplane 
Sj* :=i 3 UQU{oo}, with dC passing to the cusps ^ 3 * The 3-cones in V tiling 

^ Although the Voronoi cones aren’t necessarily simplicial, we can assume that they 
have been F-equivariantly subdivided. 
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C pass to the SL 2 (Z)-translates of the ideal triangle with vertices at 0, 1, oo. Let 
us call these ideal triangles Voronoi triangles. 

If ^ e is dual to a class in H^{r) and is supported on one 2-cone, then 
supp ^ passes to a geodesic ^ between two cusps ui, (Figure 2). We can 
subdivide /i into geodesic segments {^i} so that the endpoints ei,Ci+i of /ii lie 
in the same or adjacent Voronoi triangles. Then we assign cusps to the Cj as 
follows. If €i is not an endpoint of then we assign any cusp c, of the Voronoi 
triangle containing Cj. Otherwise, if Cj = u\ or U 2 and hence is an endpoint of /i, 
then we assign Cj to itself. This determines a homology between ^ and a chain 
supported on cones passing to the segments [ci,Ci+i]. These cones are Voronoi 
cones, and thus G CY ■ 




Fig. 2. A subdivision of n; the solid dots are the Cj. Since the lie in the same or 
adjacent Voronoi triangles, we can assign cusps to them to construct a homology 
to a cycle in CY ■ 



6 Well-Rounded Retracts 
6.1 

To conclude this article, we describe unpublished work of MacPherson and Mc- 
Connell [16] that allows one to compute the Hecke action on those F for which a 
well-rounded retraet W is available. Again for simplicity we focus on T C SL„(Z); 
our first task is to explain what W is. 

Let V = M” with the standard inner product preserved by SO(n), and let 
L C V be a lattice. For any v G V, write ||f|| for the length of v. Let m{L) be 
the minimal nonzero length attained by any vector in L, and let M{L) = {u G 
L I ||u|| = m{L)}. Then L is said to be well-rounded if M{L) spans V. 

6.2 

Consider the space of cosets Y = SL„(Z)\SL„(R). This space can be interpreted 
as the space of oriented lattices in M” modulo homotheties. Let IF C V be the 
subset of well-rounded lattices, and for any j = 0, . . . , n, let Yj = {L G Y \ 
dim span M{L) > j}. Clearly Yq = Y and F„ = W. 
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According to Ash [1], there is an SO (n)-equi variant retraction r:Y ^ W 
constructed as follows. Let L G Yj, and write V = Vi 0 V 2 , where Vi = 
(span M{L)) 0M, and V 2 is the orthogonal complement of Vi. For 0 < A < 1, let 
T(A) be the linear transformation (ui, V 2 ) (ui, AU 2 ), and let L[A] be the image 
of L under T(A). There is a critical value Aq for which dimspan M{L[X\) > j. 
Then we can define rj\Yj Yj+i by rj{L) = L[Xo\. These retractions can be 
composed to define the retraction r:Y ^ W, and the space W is the well- 
rounded retract. 

Since r is SO(n)-equivariant, it induces a retraction SL„(Z)\SL„(R)/SO(n) 
^ VF/SO(n). Moreover, W can be given the structure of a locally-finite regular 
cell-complex. In a certain sense, these cells are dual to the Voronoi cones from §5: 
Voronoi cones of codimension k are in bijection with VF-cells of dimension k. The 
construction works if F is replaced with any finite-index subgroup of SL„(Z), 
and hence one has a convenient topological model to study the cohomology of 
any such F . 

6.3 

Now we consider how the ideas used in the construction of W can be applied to 
compute the action of the Hecke operators on cohomology. Let d = (di, . . . , d„) 
be a tuple of strictly positive integers, and let g{d) G GL„(Q) be the diagonal 
matrix with entries d. Let F' := FC\g~^ Fg. The Hecke correspondence associated 
to this data is the diagram (ci,C 2 ):T'\A — > F\X, where the two maps are 
defined by ci(F'x) = Fx and C 2 {F'x) = Fgx. In terms of the above description, 

0 C 2 is the (multivalued) map that takes any lattice L to the set of sublattices 
{M C L I L/M = %ld\L®- ■ ■®T^ldrJX\. A Hecke correspondence induces a map 
c* o ( 02 )* on cohomology that is exactly a classical Hecke operator. For example, 
if n = 2, p is a prime, and d = (l,p), then the induced Hecke operator is the 
usual Tp. 



6.4 

Fix a tuple d and a pair of lattices M C L as above. Choose u G [l,oo). For 
u G L, let II II „ be ||u|| if u G M, and u ■ ||u|| otherwise. Now we can consider 
the retraction r described in §6.2, but using || ||„ instead of || || as the notion of 
length. When u= 1, the result is the usual retract W. But for u = uq sufficiently 
large, only vectors in M will be detected in the retraction. Since M is itself a 
lattice, we have Wu^ = W. 

These two complexes Wi and Wug appear in a larger complex W that depends 
on n and d and is fibered over the interval [1, uq] with fiber VF„. The fibers Wi 
and Wug map to W by the maps ci and C 2 , respectively. One computes the 
action of the Hecke operator by lifting a class on F\W to F'\W, pushing the 
lift across F'\W to the face F\Wug, and then pushing down via C 2 to T\IF. 
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Abstract. The goal of this paper is to describe a practical and efficient 
algorithm for computing in the Jacobian of a large class of algebraic 
curves over a finite field. For elliptic and hyperelliptic curves, there ex- 
ists an algorithm for performing Jacobian group arithmetic in O(g^) 
operations in the base field, where g is the genus of a curve. The main 
problem in this paper is whether there exists a method to perform the 
arithmetic in more general curves. Galbraith, Paulus, and Smart pro- 
posed an algorithm to complete the arithmetic in O(g^) operations in 
the base field for the so-called superelliptic curves. We generalize the 
algorithm to the class of Cab curves, which includes superelliptic curves 
as a special case. Furthermore, in the case of Cab curves, we show that 
the proposed algorithm is not just general but more efficient than the 
previous algorithm as a parameter a in Cab curves grows large. 
Keywords: Discrete logarithm problem, algebraic curve cryptography, 
Jacobian group, ideal class group, superelliptic curves. Cab curves 



1 Introduction 

This paper is motivated by cryptography based on the intractability of the dis- 
crete logarithm problem (DLP) in the divisor class group of a curve. While 
elliptic curve cryptography has drawn considerable public attention in recent 
years, cryptosystems using hyperelliptic curves are currently getting accepted as 
well, which seems to be based on the following considerations: 

1. the order of a Jacobian group can be large compared to the size of the field 
if the genus g of the curve is large (the Hasse-Weil bound [15]); 

2. a novel method for solving the elliptic curve DLP that would be proposed 
in the future may not be applied to non-elliptic curves; and 

3. recently, several fast algorithms for performing arithmetic on hyperelliptic 
curves have been proposed. 

For elliptic curves, a method for performing addition among Jacobians has 
been known from a long ago, and its group arithmetic is given as a simple 
formula [13] . On the other hand, an efficient method of Jacobian group arithmetic 
for hyperelliptic curves has been given by D.G. Cantor [2]. (Although Cantor 
assumed the characteristic is not two, N. Koblitz recently excluded the constraint 
[7] .) The only problem in addition of divisor classes is to compute good prescribed 
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representatives of a class. In the case of hyperelliptic curves, following Cantor 

[2] several methods for this have been proposed (see N. Smart [14] for details), 
and the algorithms realized in O(g^) operations in the base field are supposed 
to be the most efficient methods thus far. 

In this paper, we address the problem whether or not there exists a method 
for performing Jacobian group arithmetic in O(g^) operations in the base field 
for more general curves than elliptic and hyperelliptic curves. 

This problem has been solved in the affirmative for a class of curves called 
superelliptic curves (Galbraith, Panins, and Smart [5]): 

b 

C/Fq:Y- = Y^a,X^ , 

i=0 

where ai G Fq, at ^ 0, a and b are coprime, and the curve is assumed to be non- 
singular as an affine plane. In superelliptic curves, a = 2 implies a hyperelliptic 
curve, and a = 2, 5 = 3 implies an elliptic curve. 

In this paper, we consider more general curves called Cab curves [9]: 

C/Fq : ^ = 0 , 

0<z<fc,0<j <a,0<az+fcj <ab 

where aij G Fq, atfi yf 0, oo.a 0, and the curve is assumed to be nonsingular 
as an affine plane. 

Previous methods for computing Jacobians [1,5] are based on the fact that 
a Jacobian group is isomorphic to the ideal class group of the coordinate ring 
Fq[x, y] with x = XmodC and y = YmodC in a canonical manner, which holds 
for Cab curves. They reduce the problem of finding a good representative for a 
divisor class to that of finding a good representative of the corresponding ideal 
class (see Section 3). 

On the other hand, Galbraith, S.Paulus, and Smart [5] reduced the problem 
of finding the representative element of each ideal class in a superelliptic curve 
to that of finding a minimal element in a lattice belonging to ideal in the ideal 
class, where the minimization is taken based on a certain metric suitable for 
superelliptic curves (see Section 4 for details), and applied an LLL-like algorithm 

[3] which ensures to find the minimal solution for this setting (S. Paulus [11]). 
In particular, in Paulus’s LLL-like algorithm, division between polynomials is 
not required, so that Galbraith et. al’s method [5] computes Jacobian group 
arithmetic in O(g^) operations in the base field (see Section 4 for details). 

S. Arita [1] reduced the problem of finding the representative element of each 
ideal class for a Cab curve to that of finding the minimal element in an ideal 
belonging to the ideal class, where the minimization is taken based on a certain 
monomial order suitable for Cab curves (see Section 5 for details), and applied 
the so-called Buchberger algorithm that computes the reduced Grobner basis. 
However, it is generally hard to evaluate the computational effort of finding a 
Grobner basis of an ideal in a strict manner. Even in Arita’s heuristic analysis, 
computing Jacobian group arithmetic is supposed to take O(g^) operations in 
the base field. 
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In this paper, we generalize Galbraith et. al’s method [5] to Cab curves, so 
that there does exist a method which performs Jacobian group arithmetic on 
Cab curves in 0{g^) operations in the base field. To this end, we first point 
out that the lattice reduction in Galbraith et. al. [5] is essentially equivalent 
to the problem of finding the minimal element in an ideal with respect to the 
Cab order. We further modify Paulus’s LLL-like algorithm for the lattice using 
a Cab curve. As a result, we prove that the modification gives a more efficient 
algorithm. Moreover, we propose an efficient method for computing the inverse 
ideal of an ideal in the coordinate ring of a Cab curve (see Section 6 for details) . 
We will see that the method proposed in [5] for computing an inverse ideal is 
specific to the case of superelliptic curves. On the other hand, it turns out that 
a certain method for computing an inverse ideal in number fields works quite 
well for function fields defined using a Cab curve. 

2 Superelliptic and Cab Curves 

The notation follows [13] [15]. 

2.1 Superelliptic Curves 

Definition 1 ([5]). A superelliptic curve defined over K is a nonsingular curve 
given as follows: 

Y-= > ( 1 ) 

0<i<b 

where Ui € K, abp yf 0, a and b are coprime, and char(K ) does not divide a. 

By definition, in elliptic and hyperelliptic curves we have a = 2, 6 = 3, and 
a = 2, 6 > 3, respectively. Then, the genus of a superelliptic curve is given [5] by 

g={a-l){b-l)/2 . (2) 



2.2 Cab Curves 

Let C be a curve defined over K with at least one iL-rational point P. Then, if 
we define Mp := {—vp{f)\f G L(ooP)}, Mp makes a unitary semigroup under 
addition. 

Definition 2 {Cab Curve). If the semi-group Mp is generated by two positive 
integers a and b with g.c.d(a, b) = 1, the pair {C, P) is said a Cab curve. 

Let (C, P) be a Cab curve. By definition, there exist functions X G L{ooP) 
and Y G L{ooP) with pole orders a and b at P, respectively. Using these two 
functions X and Y, we obtain the affine model of the Cab curve as follows [9] : 

C/K : OiijX^Y^ = 0 , 

<a,az+fcj <afc 



(3) 
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where Ofij G K, abfi ^ 0, and ao,a ^ 0. The affine model in (3) is said the Miura 
canonical form of the Cab curve (C, P). In the Miura canonical form, a Cab curve 
is assumed to be nonsingular in the affine plane, and P is the only infinite place 
Poo of curve C [9] . 

We assume that a Cab curve is given in a Miura canonical form. Then, it turns 
out that Cab curves include superelliptic curves with the same (a, b). In fact, 
superelliptic curves are Cab curves with ajj- = 0 (0 < z < 6 and I) 

and = 0 (I<z< 6). 

As for superelliptic curves, the formula 

9={a-l){b-l)/2 (4) 



holds also for Cab curves. 

Definition 3 (Cab Order [9]). We order as a >ab P for a = {ai,a 2 ),P = 
(PitP 2 ) G ZyQ if one of the following two conditions holds: 

1. aa\ + bu 2 > aPi + 6/J2 , or 

2. aai + ba2 = a/Ji + 6 /J 2 , cxi < Pi . 

By definition, under the condition C(x, y) = 0, monomials are ordered 

based on the pole order at infinity Poo'- 

—vp^ = aai + 602 , 

and if they are equal, we suppose that the larger the degree with respect to X, 
the smaller the monomial order. 

Similarly, polynomials / = X) be ordered according to the pole 

order at infinity Poo- 



-'op^ if) = maxij_Q, ^^o{az + bj}. 

3 Isomorphism between Jacobian and Ideal Class Groups 

Jacobian group arithmetic on Cab can be realized using the fact that the Jacobian 
group is isomorphic to the ideal class group of the coordinate ring for superelliptic 
and Cab curves [1,5]. 

Definition 4. If D G Div^(C) is expressed as E — nPoo with E > 0 and 
Poo ^ support (if), D is said a semi-reduced divisor. 

Lemma 1 ([1,5]). For each j G Jk(C), there exists a semi-reduced divisor 
D G Div^(C) such that j = [D], 



Definition 5. If n is minimized in Di = E — nPoo with E > 0 and Poo ^ 
support(if) (semi-reduced) and Di ~ D G Div^(C), then D\ is said the reduced 
divisor equivalent to D. 
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Lemma 2 ([1,5]). If D = E — nPoo G Div’^{C) with E > 0 and Poo ^ 
support(L^) is a reduced divisor, then the reduced divisor D\ ^ D is unique 
for each D G Div^{C), and deg{E) < g 

We can obtain reduced divisors using the following algorithm [1,5] 

Algorithm 1. 

Input: Semi-reduced divisor D = E — nPoo G Div'j^{C) with E > 0 and Poo ^ 
support (if). 

Output: The reduced divisor G ~ —D. 

Step 1: Find f G L{ooPoo) satisfying (/)o > E and the pole order —vp^{f) is 
minimal, where L{ooPoo) '■= U]^jf’L(iPoo)- 
Step 2: G^-D+{f). 

Since Algorithm 1 outputs a divisor equivalent to (—1) times the input divisor, 
if Algorithm 1 is applied twice, a divisor equivalent to the input divisor can be 
obtained. 

However, directly dealing with divisors is not generally efficient because of 
irreducible decomposition of polynomials. So, Arita [1] and Galbraith et. al. [5] 
independently proposed Jacobian group arithmetic using ideal representation. 

Since Gab curve (C, Poo) is nonsingular in the affine plane, the coordinate 
ring K[x,y\ with G{x,y) = 0 is a Dedekind domain. For a Gab curve (C, Poo), 
an isomorphism between the Jacobian group Jk{G) and the ideal class group 
H{K[x, y]) of K[x, y] is given as follows: 

: Jk{G) ^ H{K[x,y]) , 

I X! npP - { ^ np)Poo] 1-^ [L(ooPoo - ^ npP)], (5) 

PeC.P^^Pcx, P^C,P^P,ya P^C,P^P,ya 

where we denote the ideal class which ideal I C K[x,y] belongs to by [/]. 

We call the ideals corresponding to reduced and semi-reduced divisors the 
reduced and semi-reduced ideals, respectively; then each semi-reduced ideal I 
is expressed by an integral ideal / = L{ooPoo — E) C L{ooPoo) = K[x, y] with 
P > 0 and Poo ^ support (P). 

Now each integral ideal of K[x, y] is a P[a;]-module, and if a P[a;]-basis is 
given as (/?o, • • • , Pa-i) with (3i = X)j=o P[ 2 :]-basis can be uniquely 

expressed by taking the Hermite normal form (HNF) of the matrix (flij) (see 
Appendix). Therefore, we express each representative element of an ideal class 
group in K[x, y] by the HNF of the P[a;]-basis. 

Defiuitiou 6. We define the degree of a (fractional) ideal in K[x,y] to be a 
degree of x in the product of the diagonal elements ( subtracted by the degree of 
the denominator) of the HNF. 

Then, it turns out that the degree of an ideal coincides with a value of n in 
the corresponding semi-reduced divisor E — nPoo- Hence, the sum of the degrees 
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with respect to x in each column of the HNF of a reduced ideal is at most g 
(see Lemma 2). It is known that the product of diagonal elements in the HNF 
expression of / is the norm of / [5] . 

Hence, Algorithm 1 can be replaced by 

Algorithm 2. 

Input: Semi-reduced ideal I. 

Output: The reduced ideal J ~ I~^ . 

Step 1: Find f & I, / yf 0 such that the pole order —vp^{f) is minimal. 

Step 2: J ^ ■ 

4 Jacobian Group Arithmetic on Superelliptic Curves 

Galbraith et. al. [5] proposed an algorithm (Algorithm 3) for performing Jacobian 
group arithmetic on superelliptic curves. Algorithm 3 below computes a K[x\- 
basis to represent an ideal in an ideal class: we embed K\x,y] into {K[x])‘^ with 

(j) : K[x,y] ^ {K[x\Y 

Ci{x)y'^ ^ {co{x)p ■ ■ ,Ca-l{x)) 

0<i<a— 1 

and define the metric of C = (cq(x), ■ ■ ■ , Ca-i(x)) G {K[x\Y as follows: ICI := 
maxICIi where \C\i := deg 2 ,(ci(a;)) + Consider an ideal / C K[x,y] and let 

{/o) • • • ) fa-i} be a A[a;]-basis of /; then, (p{I) is a lattice generated by {(p{fi)}i 
over A [a;], so that minimization over f € I with respect to —vp^ (/) is equivalent 
to minimization over u G 4>{I) with respect to |u| {—vp^ (/) = a\4>{f)\ for / G I). 
Galbraith et. al. [5] apply Paulus’s method [11] in the following way. 

Definition 7 ([5]). The orthogonality defect OD{fo, • • • , fa-i) of a basis {fo, 

■ ■ ■, fa-i} for a lattice L is defined as 

OD{fo, • • • , fa-i) := y] |/,| - degYd{L)), 

i 

where d{L) := det(/o*, • • • , f*_Y with f* := {Y{x), fi(x)x^ • • • , fi_^x^^^~^'>y 
for fi = X;“=o fj(x)yT 

It is easy to see that OD{fo, ■ ■ ■ , fa-i) > 0. 

Definition 8 ([11]). The basis {/o, • • • , /a-i} for a lattice is said a reduced 
basis if OD{fo, • • • , fa-i) = 0. 



Proposition 1 ([11]). Let {fo, • • • , fa-i} be the reduced basis for an lattice L. 
Then f G {/o, • • • , /a-i} such that \f\ = mini{|/i|} is the minimal nonzero 
element in L with respect to | • | . 
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Algorithm 3 (Jacobian Group Arithmetic on Superelliptic Curves [5]). 



Input: Reduced ideals I\, I 2 in K[x,y] (HNF). 

Output: The reduced ideal I 3 ~ I 1 I 2 (HNF). 

Step 1: D ^ I 1 I 2 ; 

Step 2: J ^ a semi-reduced ideal equivalent to D~^ ; 

Step 3: f ^ a minimal nonzero element in J with respect to |</'(-)l- 
Step 4: I 3 ^ the HNF of 

The validity of Algorithm 3 can be easily checked: basically, the process of 
Algorithm 2 is done twice in Steps 2-4. (Note that J in Step 2 is not required to 
be a reduced ideal but I 3 , in Step 4 is.) We now discuss some of theses steps in 
detail; this will show that Algorithm 3 really uses superelliptic curves. 

In Step 2, for D = I 1 I 2 an integral ideal equivalent to D~^ is computed using 
the formula 

D ^ ~ H„^Gal(K(x,y)/K(x)),a^lD'^ ■ 

Note that here it is assumed that K contains the a-th roots of unity. So if 
necessary, the base field is extended in this step. Any a G Gal{K{x, y)/K{x)) is 
given by y"^ = py for some a-th root of unity p. Hence the conjugates D'^ and 
therefore also D~^ are easy to compute. It seems unclear how to extend this 
idea to more general Cab curves. 

For Step 3, we can obtain the minimal element by finding the reduced basis. 
The complexity of finding a reduced basis is given as follows: 

Propositiou 2 ([11]). We can find the reduced basis from a K[x]-hasis {Co, 

• • •, Ca-i} of the lattice in 

0(a^max|Ci| x OD{Cq, • • • , Ca_i) log^ q). (6) 



For Step 4, since 



h = J~Hf) 



D 



l\D- 



if) 



I1I2 

^K(x,y)/K(x){hl2) 



if) , 



and since the norm Nx(x,y)/K(x)ihl 2 ) is obtained computing the product of 
the diagonal elements in the HNF of the ideal I 1 I 2 , the ideal can be easily 
computed [5]. 

In summary, the whole computation can be evaluated as in Proposition 3. 



Proposition 3 ([5]). Let CjK be a superelliptic curve. Jacobian group arith- 
metic on JacKiC) (Algorithm 3) can be performed in 0{a'^ g^log^ q) if a\q — 1 
and in 0 {a^g^ log^ q) if a J(q — I 
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5 Jacobian Group Arithmetic on Cab Curves 

Arita[l] proposed an algorithm (Algorithm 4) for performing Jacobian group 
arithmetic on Cab curves. Algorithm 4 below computes a, K[x, y] -basis to repre- 
sent a unique ideal in an ideal class. The idea is that in Cab order, monomials 
are arranged according to the pole orders at infinity Poo when they are regarded 
as functions on a Cab curve. 

Algorithm 4 (Jacobian Group Arithmetic on Cab Curves [1]). 



Input: Reduced ideals I\, I 2 in K[x,y\. 

Output: The reduced ideal I 3 equivalent to ideal product Iil 2 - 
Step 1: J ^ hh; 

Step 2: / ^ the minimal nonzero element in J with respect to Cab order; 
Step 3: h ^ the minimal nonzero element with respect to Cab order satisfying 

(h)J C (/); 

Step 4: I 3 ^ {h/f)J. 

The validity of Algorithm 4 can be easily checked: basically, the process of 
Algorithm 2 is done in Steps 2-4. (In particular, h and play the roles of 

the / and / in the second round of Algorithm 2, respectively.) 

In Algorithm 4, the minimal element in an ideal is computed by finding the 
reduced Grobner basis. (Note that a reduced Grobner basis gives the unique 
representation of an ideal.) However, it takes much time to obtain a Grobner 
basis, and it is hard to evaluate its computational effort in a strict manner. In 
[1], the computation of Step 2 is heuristically analyzed to be 0(g^log^ q) if the 
value of a is bounded. 

However, to authors’ knowledge, it seems that there has been none thus far 
to address Jacobian group arithmetic on Cab curves except Algorithm 4 [1]. 

6 Fast Jacobian Group Arithmetic on Cab Curves 

From the considerations in the previous sections, it turns out that the following 
two problems should be solved for extending Galbraith et. al.’s method to Cab 
curves: 

1. how to compute the inverse ideal I~^ given an ideal /; and 

2. how to compute the minimal element over an ideal with respect to Cab order. 

6.1 Computing Inverse Ideals 

For the first problem, we propose a more general method to obtain an inverse 
ideal than that in the case of superelliptic curves. The idea is based on the 
method for computing inverse ideals in the integral closure of a number field [3] . 
Let L be a number field, and Zl the integral closure of L, and n := [L : Q], We 
first fix the Z-basis (wi)i<i<n of Z^. 
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Definition 9. The different of L is defined as 

T{L) := {x G L\TY&ceL,Q{,xZL) C Z}"! . (7) 

Then, the following proposition follows [3] : 

Proposition 4. Let {u>i)i<i<n be a Z-hasis of Zl and I an ideal of Zl given 
by a matrix M whose columns give the coordinates of a Z -basis (7i)i<i<n of 
I on the chosen Z -basis. Let T = (Uj) be the n x n matrix such that Uj = 
TraceL/Q^uJitOj). Then, the columns of the matrix (M*T)~^ form a Z -basis of 
the ideal {LT{L))~^ . 

Therefore, for a given ideal / C Zl, the ideal product LT{L)~^ is computed by 
taking the HNF of the nxn^ matrix obtained from M and . If the HNF is N, 
then, by Proposition 4, (N*T)~^ forms a Z-basis of {LT{L)~^)~^ T{L)~^ = L~^ . 

Now we go back to the case of Cab curves. The ring L(ooPoo) is a Dedekind 
domain. Furthermore, since Cab curves are generally nonsingular, L(ooPoo) co- 
incides with the coordinate ring K[x,y\. Therefore, the integral closure of K[x] 
in K{x,y) is K[x,y], so that the result for Zl can be extended to K[x,y] in 
a natural manner. Then, l,y, can be the A'[a;]-basis of K[x,y], and 

T = {tij)i<i<a, i<j<a are given by Uj = Tra,ceK(x,y)/K(x){y'‘''~^~‘^)- The value of 
each tij can be computed using the Newton formula (page 163, [3]) if the defini- 
tion equation is given. Let Di{x) and as Di{x)y^ (the definition 

equation of a Cab curve) and y* = (a < * < 2a — 2), respectively, 

in K[x, y] with x = X mod C and y = Y mod C. Then, Tra,ceK{x,y)/K(x){^) = 
Trace K(x,y)/K{x){y) = Da-i{x), for z = 2, • • • , a - 1 



i-l 

Tracex(x,y)/K{x)(,y ^ — ihda—i{x) -h ^ dD a— i{x')Tr ace x{x,y) / K{x)^y ) ( 8 ) 

i=i 



and for i = a, ■ ■ ■ ,2a — 2 



a— 1 

TraCex(a:,y)/iC(a:)(y ) — ^ ^ C^ (.x')TvaCej.[ (^x ,y) / K (x){y ) ■ (9) 

1=0 

If we compute and store the matrix dT~^ with d the determinant of T before- 
hand, we obtain: 

Algorithm 5 (Computation of Inverse Ideals for Cab Curves). 



Input: Semi-reduced ideal L in K[x,y] with {'ji)i<i<a a K[x]-basis of L (HNF). 
Output: The inverse ideal L~^ . 

Step 1 : N ^ the HNF of the a x of matrix (7i<5j), with Sj column vectors of 
dT~^; 
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Step 2i h ^ det(-/V*); 

P ^ dh{N*T)-^ = (dT-^){h{N*)-^); 
k ^ GCM(GCM(P), h); 
e ^ • 

W ^ iP; 

/-I ^ (VR,e) ('/-i = W(e)-^J. 

(GGM(^) with A a matrix and GGM(/, g) with f, g G K[x] denote the GGM 
of all the elements in A and that of / and g, respectively.) 

Theorem 1 . Algorithm 5 is computed in 0{aP’g^ log^ q) and in 0{a‘^g^ log^ q) 
for Cab and superelliptic curves, respectively, if the degree of an ideal I is 0{g). 

Theorem 1 is obtained based on the following facts: if the degree of x in the 
determinant of an m x n matrix M is bounded by t, 

1 . the Hermite normal form (HNF) of M with rank(M) = m is obtained in 

0{mfnt'^ log^ q) (for the proof, see Appendix); 

2. if n = m, the determinant of M is obtained in 0(max{m^t log^ q, t^log^ q}) 

(for the proof, see Appendix); 

3. if n = m, the inverse of M is obtained in 0(max{m®t log^ g, log^ g}). 

(computing the mf determinants yields the inverse ideal if Gramer’s formula 
is applied); 

and if the degrees of x in two polynomials /, g is bounded by s, 

4. the GGM of / and g is obtained in 0(s^ log^ q). 

Proof of Theorem 1: 

1 ) General case 

For Step 1, the degree of x in Tra,ceK{x,y)/K{x){y^), 0 < z < a — 1, is 0{g): in 
fact, from degx[Da-i{x)] < b, 0 < I < a — 1, and (8), we have 

degx[TraceK{x,y)/K{x){y")] < Taiax{degx[Da-i{x)]+degx[TraceK{x,y)/K{x){y''~'')]} 

l<Z<z 

< max{b + degx[TraceK{x,y)/K{x){y"~'^)]} 

l<Z<z 

<ib + degx[TAaceK(x,y)/K{x){y°)] 

= ib . 

For a < i < 2a — 2, one checks that the degree of x in C^^^x) is at most 
6(z— a+1) (In fact, degx{y°') = b, and degx{y'‘) < 6(z— a+1) implies dega;(?/*+^) < 
6(z — a + 1) + 6 for z = a + 1, • • • , 2a — 2.), so that 

degx[PraceK(x,y)/K(x){y'')] < max {degx[c[''\x)]+degx[PraceK(x,y)/K(x){y'^)]} 

0<Z<a— 1 

< max {b{i — a + 1) + ^6} 

0<Z<a— 1 

< ib , 
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where (9) has been applied. 

In any case, the degree of x in each element of T is ib = 0{g) (see (4)). If we 
apply Cramer’s formula, the degree of x of each element in dT~^ with d = det(T) 
is bounded by 0{ag) since the degree of x of each element in T is at most g, so 
is deg„((5j). 

On the other hand, by assumption the degree of x in each element in the HNF 
expressing the input ideal is at most g, i.e, deg 2 ,( 7 j) < g. Since there are pairs 
of they are obtained in 0{a^ g^ log^ q). Using 1 with m = a, n = a?, 

and t = 0{a?g), the HNF N of the ax a? matrix is obtained in 0{a^g^ log^ q). 

For Step 2, if we apply Cramer’s formula, and det(ft-) are computed 

in 0{aJ g"^ log^ q) (use 3 and 2 with m = a and t = O(a^g), respectively). Since 
the degrees of x of each element in matrices dT~^ and are 0(ag) and 

O(a'^g) (note that the degree of x in each element of an HNF is at most a times 
as that of the original matrix), the degree of x in each element of matrix P 
is 0{a‘^g). Since the GCM of two polynomials of degree 0{a‘^g) is computed 
in 0{a‘^g‘^log^ q) (use 4 with s = O(a^g)), GCM{GCM{P),h) is computed in 

0(aV log^?)- 

Since divisions between polynomials of degree 0{a‘^g) are done (recall that 
the degree of x in each element of P is 0{a‘^g), so is the degree of x in k), W is 
obtained in 0(a®g^ log^ q). 

Hence, Step 2 takes 0{a^g'^ log^ q). 

Therefore, Algorithm 5 takes 0(a®g^ log^ q). 

2) the case of superelliptic curves 

Let = f{x) be a Gab curve with dega;f{x) = b. 

For Step 1, the HNF representation of the ideal dT~^ is [f{x), y, • • • , 

In fact, one checks 



a 


0 


0 • 


■ • O' 




r m 0 0 • 


■ - o' 


0 


0 


0 • 


■ • af{x) 




0 


0 0- 


-- 1 










1 

II 








0 


0 


af{x) ■ 


■■ 0 




0 


Ol- 


■ - 0 


_0 


af{x) 


0 • 


■ • 0 _ 




0 


io- 


■ - 0 _ 



and d = a“(/(a;))“ Thus, the degree of the ideal expressed by dT ^ is 
degxfix) = b since the HNF of dT~^ is 

7(a;)0 0---0' 

0 1 0 

0 : : : 

0 0 0 • • • 0 
0 0 0 • • • 1 _ 

and the total degree of the diagonal elements is degxf{x) = b. On the other 
hand, by assumption, the degree of the ideal expressed by (y^) is 0{g). So, the 
degree of the ideal expressed by the HNF N is 0{g) + b = 0{g). Since 
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are obtained in 0{a^g^log^q), from 1 with m = a, n = a?, and t = 0{g), 
the HNF N is obtained in 0{a‘^g^log^ q). And, the degree of the ideal N is 
0{g + b) = 0{g). 

For Step 2, if we apply Cramer’s formula, (-/V*) ^ and det(h) are computed 
in 0(max{a®glog^ g, log^ g}) = 0(a^g^ log^g) (use 3 and 2 with m = a 
and t = 0{g), respectively, and note a = 0{g)). Since the degrees of x of 
each element in matrices dT~^ and are 0{g), the degree of x in each 

element of matrix P is 0{g). Since the GCM of two polynomials of degree 0{g) is 
computed in 0(g^ log^ q) (use 4 with s = 0(g)), GCM(GCM(P), h) is computed 
in 0(a^g'^ log^ q). 

Since divisions between polynomials of degree 0(g) are done (recall that 
the degree of x in each element of P is 0(g), so is the degree of a; in fc), W is 
obtained in 0(a^g‘^ log^ q). 

Hence, Step 2 takes 0(a^g^ log^ q). 

Therefore, Algorithm 5 takes 0(a^g^ log^ q). □ 

Note that in the proof of Theorem 1, degree of x in each element of W is 
bounded by 0(a?g) and 0(g) for Gab and superelliptic curves, respectively, which 
will be referred later. 



6.2 Computing the Minimal Element 

For the second problem, by the definition of the metric | • | in (AT[a;])“, for 
/ = E“=o ^ K[x,y] with fi(x) € K[x], we have 

-vp,^(f) = maxi{adeg,,,(fi(x)) + bi} = a\(p(f)\ . 

Therefore, for an ideal / C K[x,y], minimization over I with respect to Gab 
order is equivalent to minimization over 4>(I) with respect to | • | , so that for the 
second problem we can apply Paulus’s method [11] (finding the reduced basis) to 
Gab curves. 

Proposition 5 ([11]). Let be a basis for a lattice L and denote by 

bij the j-th coordinate of bi. If the coordinates of the vectors 5i, • • • , can be 
permuted in such a way that they satisfy 

1. |6ij < j5j] for 1 < i < j < n; and 

2. \bij\ < \bi^i\ > \bi^k\ for 1 < j < i < k < n. 

Then 6i, • • • , forms a reduced basis. 

Now we go back to the case of Gab curves. For a A'[a;]-basis /o, • • • , fa-i for a 
lattice L, if it satisfies that \fi\ — \fj\^Z (0 < i < j < a — 1), then /o, • • • , fa-i 
forms a reduced basis by Proposition 5. (Note that g.c.d(a, b) = 1 implies there 
exists an unique I such that ]/] = ]/]/ for a nonzero vector / = (/q, • • • , fa-i) G 
(K[x])°'. In fact, if ]/] = \f\i = \f\j with 0<i<j<a — 1, i.e. aci + bi = 
acj + bj, where c, and Cj are the degrees of x in fi(x) and fj(x), respectively. 
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then a{ci — cj) = b{j — i). Hence, a\j — i since g.c.d{a,b) = 1, which implies 

i = j-) 

Therefore, we can modify Paulus’s algorithm [11] to obtain the following 
algorithm. 

Algorithm 6. (Computation of Reduced Basis in Cab Curves) 

Input: K[x]-basis {/o, • • • , /a-i} for a lattice L with fi = ■■■, 

Output: The reduced basis. 

Step 1: 50 ^ fo, k ^ 1; 

Step 2: 5 fc ^ /a,; 

Step 3: if | 5 j| — | 5 fc| ^ Z (Vj < k) then k ^ fc + 1, otherwise go to Step 5-1; 
Step 4 if k = a then output {go, • • • , 5a-i}, otherwise go to Step 2; 

Step 5-1 letj, I be the indices such that |5j| — |5fc| G Z, \gj\ = \gj\i, \gk\ = \gk\ii 
Step 5-2 if \gj\ > \gk\ then swap gj and gki 

Step 5-3 5fc ^ gk — with r = Ck,i/cj^i, where Ck,i and Cj^i are the 

leading coefficients of gk,i{x) and gjj{x), respectively; and 
Step 6 if Yf]=o \9j\ + E“=fc+i \fj\ = deg„,(i(T) then output {go,---,gk, /fc+i, 

• • •, fa-i}, otherwise go to Step 3. 

The validity of Algorithm 6 can be checked by Definition 8 and Proposition 
5. 

Theorem 2. Algorithm 6 is computed in 0{aH{t + b) log^ q) if the degree of x 
in {fij)ij is bounded by t. 

Proof of Theorem 2: 

It is easy to check that Step 5-3 dominates the computational complexity of 
Algorithm 6. In Step 5-3, the computation of gk ^ gk — requires 

shift operations and 0{at) multiplications in K. Note that OD{go, ■ ■ ■ ,gk, /fc-i-i, 

• • •, fa-i) strictly decreases after executing Step 5-3. Therefore, the number of it- 
erations of executing Step 5-3 is bounded by a x {OD{fo, ■ ■ ■ fa-i) —deg^d{L)) < 
a X OD{fo, ■ ■ ■ fa-i) = 0{a{J2iZo{t + b))) = 0(a^(t J- b)). Hence, Algorithm 6 
is computed in 0(at log^ q x 0(a^(t -I- b))) = 0{aAt{t J- b) log^ q) □ 

For Steps 3 and 5, in [5], Paulus’s original algorithm was directly applied in 
a straightforward manner that a set of linear equations 

fc-i 

XI = 4,i (0 < i < fc - 1) 

i=o 

is solved for Vj, j = 0, • • • , fc— 1, every time fc and OD{fo, • • • , fa-i) are updated, 
where is the leading coefficient of gj,i{x) at the order of the leading term of 
in gj with respect to Cab order (if no such a coefficient exists in gj^ifx), then 
c* ^ = 0) and = 0 for 0 < j < z < fc — 1 (if necessary, swap rows in each gj), 
so that the leading term in gk can be cancelled out with either of go, - ■ ■ ,gk-i- 
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They estimated the complexity of solving the equations as O(fc^) operations in 
the base field since the coefficient matrix (c* ^)o<i,j<fc-i is a lower triangular 
matrix (thus, that of computing a reduced basis as 0{a^ g^log^ q)), which we 
considered too large for implementation. In this paper, we find that the solution 
is quite simple, i.e. we only solve one linear equation (see Step 5-3) since all rj 
except one are equal to zeros, which is complited in 0(1) operations in the base 
field. In fact, we have 

1. for each column in the coefficient matrix (cj^j)o<ij<fc-i and the column 
vector (cfc,i)o<i<fc-i, all the elements except one are equal to zeros; and 

2. for each row in the coefficient matrix ad the elements except 

one are equal to zeros. 

(Apparently, cj^i = c* j c* j yf 0 \gj\ = \gj\i), where the first property is 
from the assumption g.c.d{a, b) = 1, and the second from Step 3 in Algorithm 
6, i.e. \gj\ - \gi\ ^ Z {0 < i < j < k - 1). 

Algorithm 6 utilizes these property, so that the computational effort has been 
greatly saved. 

In summary, we see that the extension of Galbraith et. al.’s method to Cab 
curves is possible. The whole proposed algorithm can be described as in Algo- 
rithm 7. 

Algorithm 7. (proposed Jacobian group arithmetic on Cab curves) 
Input: Ideals I\, I2 in K[x,y] (HNF). 

Output: The reduced ideal I3 equivalent to I1I2 (HNF). 

Step 1: J ^ the HNF of I 1 I 2 ; 

Step 2: Applying Algorithm 5 to J, J~^ <— (IF,e); 

Step 3: Applying Algorithm 6 to W , / <— the minimal element in W with 
respect to Cab order; 

Step 4: /a ^ the HNF of {f)W~^ = {f/e)J. 

Our final task is to ensure that Algorithm 7 is completed in O(g^) operations 
in the base field if the sizes of a and q are bounded, which is the goal of this 
paper. The computation time of Steps 1 and 4 is basically the same as those 
in Algorithm 3, which is within O(g^) operations in the base field. Step 2 is 
completed in 0{a^g^log^ q) by Theorem 1 since the degree of ideal J is 0{g). 
For Step 3, since the degree of x in each element of the matrix W is 0{a‘^g), 
Step 3 is completed in 0{a^ ■ afg ■ {afg + b) ■ log^ q) = 0{aJg'^ log^ q) by Theorem 
2. Hence we obtain: 

Theorem 3. Algorithm 1 is completed in 0(a®g^ log^ q) and in 0{a‘^g'^ log^ q) 
for Cab and superelliptic curves, respectively. 

(See Table 1 for details.) 
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Table 1. Complexity of Jacobian Group Arithmetic 





Proposed method 


Galbraith, Panins 
and Smart [5] 


Cab 


superelliptic 


superelliptic 


Step 1 

(ideal product) 


0{a*g'^ log^ q) 


0{af^q^ log^ q) 


0{af^q^ log^ q) 


Step 2 

(inverse ideal) 


0{a^g^ log^ q) 


Oiaf^q^ log^ q) 


Oia’q^ log^ q) 
{0{a^q^ log^ q)) 


Step 3 

(minimal element) 


Oia’g^ log^ q) 
(substitute t = af'q 
to Theorem 2) 


0{a^q^ log^ q) 
(substitute t — q 
to Theorem 2) 


0{aJq^ log^ q) 
(apply Proposition 2) 


Step 4 

(ideal product) 


Oia’q^ log^ q) 


log^ q) 


0{af^q^ log^ q) 


whole process 


0{a^q^ log^ q) 


0{a!^q^ log^ q) 


0{a‘q^ log"" q) 
{o[a^q^ log^ q)) 



7 Concluding Remarks 

We proposed a fast Jaconbian group arithmetic algorithm for Cab curves (Al- 
gorithm 7), evaluated the complexity of the proposed algorithm. As a result, it 
turned out that Algorithm 7 is more efficient than Algorithms 3 (Galbraith et. 
al.) in the case of superelliptic curves (Proposition 3, Theorem 3). Furthermore, 
although Algorithm 7 can be applied to Cab curves as well as superelliptic curves. 
Algorithm 7 completes the arithmetic in 0{g^) operations in the base field while 
Algorithm 4 does in 0{g^) operations in the base field. 

Future work includes exploring a faster Jacobian group arithmetic scheme 
for more general curves. 
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Appendix : Hermite Normal Form (HNF) with K[x] 
Coefficients 

Definition 10. We say that an m x n matrix A = (aij) with K[x] coefficients 
is an Hermite normal form (HNF) if there exists r <n and a strictly increasing 
map f from [r 1, n] to [1, m] satisfying the following properties: 

1. /or r J- 1 < j < n, 0, Uij = 0 if i > /(/); and for k < j 

(a) deg,^(a/(fc),j) < deg,^(a/(fc).fc) z/deg,^(a/(fc),fc) > 1; or 

(b) af(k),j = 0 z/deg,j,(a/(fc)_fc) = 0 (equivalently, af(k),k G K) 

2. the first r columns of A are equal to 0. 

3- o/(fc),fc; k = r 1, ■ ■ ■ , n, are monic. 

Proposition 6. Let A = (aij) he an mxn matrix with K[x] coefficients. Then, 
there exists a unique mxn matrix B in HNF of the form B = AU with U € 
GLn{K[x\) , where GLn{K[x\) is the group of matrices with K[x] coefficients 
which are invertible, i.e. whose determinant belongs to K. 

We call the matrix consisting of the last n — r columns the HNF of A. 

When we compute an HNF directly, it is hard to evaluate its complexity 
since we don’t know how large the degree of x grows during the process. But, 
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in the case of integer coefficients and rank (A) = m, if we know the value D 
that is a multiple of the determinant of the Z-module L{A) generated by the 
columns of A, then we can compute the HNF of A by using D [3]. And this 
modified method requires 0{m^n\D\^)-hit operations, where \D\ is the number 
of bits for expressing D. (Note that in the case of a finite field, the computation 
of an HNF takes O(m^n) operations in the field [3].) Therefore we obtain the 
following algorithm by extending the result for Z to K[x\ in a natural manner. 

Algorithm 8. HNF 

Input: m X n matrix A with K[x] coefficients and rank(A) = m. 

Output: The HNF of A. 

Step 1: the R ^ the mxm matrix whose columns consist of linear independent 
column vectors of A; 

Step 2: D ^ det(i?); 

Step 3: Compute the HNF modulus D [3]; 

Remark 1. 1. In the case of m = n, Step 1 is not required. 

2. In Step 2, since L{R) is an sub-module of L{A), the value of D is a multiple 
of det{L{A)) , where {L{A)) is a K[x]-module generated by the columns of A. 

Proposition 7. We assume the degree ofx in the determinant of A is less than 
t. If q > t, then Algorithm 8 is completed in Oijnfnf^ log^ q). 



Remark 2. We consider the ease where g is extraordinarily large, say q = 
(common in cryptography etc.), so that the condition q > t is always cleared. 
Otherwise, no computational problem arises. 

Proof: 

For Step 1, let ai, • • • , a„ be the column vectors which A consists of, and 
Ai = be the matrix that consists of the first i columns of A. We 

consider W <Z K oi cardinality t (such a W always exists because #(VF) = t < 
q = ff{K)). Then, we have 

deg,^(det(L(A))) < t = deg^^iHaewU) ■ (10) 

Let rffii) := rank/f[2,]/(y^(a;))(Ai mod fffix)). Then, we can show that there 
exists an fffix) such that rank(A) = rffin). Suppose rank(A) < rffin) for all 
a & W. Then, det(L(A)) mod /« = 0 for all a & W. But, this implies Ha^wfa 
divides det(L(A)), which contradics (10). 

So, we can construct linear independent column vectors of A, i.e. Step 1 can 
be broken down into the following stages: 

Stage 1 choose an /„ G W, and for each 1 < i < n compute rffii)] 

Stage 2 if there exists an I such that Ta{l) = m, go to Stage 4-1; 

Stage 3 W ^ W — {fa} and go to Stage 1; 

Stage 4-1 if rc(l) = 1, then choose oi, otherwise throw away oi; and 
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Stage 4-2 for each 2 < i < I, choose at such that Vaii — 1) < r{i). 

It is clear that the computation of Stage 1 dominates Step 1. We can ob- 
tain the value of ra(z), 1 < i < n, by computing the HNF of the a x 
matrix ^mod/a(a;). From K[x]/{fa{x)) = K and the fact that the number 
of iterations in Stage 1 is bounded by #(VF), it turns out that Step 1 takes 
#(VF) X 0{rn?"n\o^ q) = 0{t) x 0{m‘^nlog^ q) = O {m'^ nt log^ q) (the HNF is 
obtained in 0{m^nlog^ q) if each element of the element is in K [3], which is 
much smaller than that for K[x]). 

For Step 2, we can obtain the value of D by computing D mod fa{x) for 
each a & W and applying the Chinese Remainde Theorem. It takes #(VF) x 
0(m^ log^g) = 0{m^tlog^ q) to compute D mod fa{x) for all a € W. Then, 
D = Eaga{D mod fa{x)), where ga = Saha with Vafa + Saha = 1, where 
Ta € K[x] and ha = na>ew fa> / fa- The multiplication Ila^wfa is done in 
■ log^ q) = log^ q); the division between Ila^wfa and fa is done 
in Oft ■ log^ q) since the degrees of x in the two polynomials are t — 1 and 1; 
Sa is computed in 0(1 • log^ q) (Proposition 3 [12]); the multiplication Saha is 
done in 0{tlog^ q) since the degrees of x in Sa and ha are 0 and t — 1; and 
the final computation Saga{D mod fa{x)) takes #(VF) x 0(1 • (t — 1) log^ q) = 
t X 0(tlog^ q) since the degrees of a; in /iq is t — 1 and D mod fa{x) G K. Hence, 
Step 2 takes 0(max{m^t log^ q, log^ q}). 

Step 3 takes 0(m^n(t log = Ofmfnt^ log^ q) [3], since the number of bits 
expressing D is Oftlogq), 

Since m < n, Algorithm 8 is completed in 0(m^nt^ log^ q). □ 
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Abstract. Essentially all subexponential time algorithms for the dis- 
crete logarithm problem over finite fields are based on the index calculus 
idea. In proposing cryptosystems based on the elliptic curve discrete loga- 
rithm problem (ECDLP) Miller [6] also gave heuristic reasoning as to why 
the index calculus idea may not extend to solve the analogous problem 
on elliptic curves. A careful analysis by Silverman and Suzuki provides 
strong theoretical and numerical evidence in support of Miller’s argu- 
ments. An alternative approach recently proposed by Silverman, dubbed 
‘xedni calculus’, for attacking the ECDLP was also shown unlikely to 
work asymptotically by Silverman himself and others in a subsequent 
analysis. The results in this paper strengthen the observations of Miller, 
Silverman and others by deriving necessary but difficult-to-satisfy con- 
ditions for index-calculus type of methods to solve the ECDLP in subex- 
ponential time. Our analysis highlights the fundamental obstruction as 
being the necessity to lift an asymptotically increasing number of ran- 
dom points on an elliptic curve over a finite field to rational points of 
reasonably bounded height on an elliptic curve over Q. This difficulty is 
underscored by the fact that a method that meets the requirement im- 
plies, by virtue of a theorem we prove, a method for constructing elliptic 
curves over Q of arbitrarily large rank. 



1 Introduction 

In the elliptic curve discrete logarithm problem (ECDLP), we are given an el- 
liptic curve E over a finite field and two points P and Q on the curve, and 
the problem is to find an integer n (if it exists) such that Q = nP. The ECDLP 
is an analog of the discrete logarithm problem over finite fields, which is the 
basis of many public key cryptosystems. Miller [6] and Koblitz [3] independently 
proposed public key cryptosystems based on the elliptic curve discrete logarithm 
problem. In proposing such cryptosystems Miller [6] also gave heuristic reasoning 
as to why the index calculus idea, which lies at the heart of all the subexponen- 
tial algorithms for the discrete logarithm problem, may not extend to solve the 
elliptic curve discrete logarithm problem. 
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The classical index calculus method for the discrete logarithm problem works 
by lifting the problem from a finite field to the ring of integers, where there is 
much richer arithmetic structure to take advantage of. To extend this idea to 
work for the ECDLP, it is natural to consider lifting an elliptic curve E/Fp of 
interest to some elliptic curve S/Q in order to possibly take advantage of the 
structure of F(Q). Miller pointed out the difficulty for such an approach is at 
least two fold: first in lifting the curve if to a curve E of sufficiently large rank 
over Q, then in actually lifting points from E to rational points of reasonably 
bounded height on F. A careful analysis by Silverman and Suzuki in [10] provides 
strong theoretical and numerical evidence in support of Miller’s arguments. 

Silverman [8] proposed an alternative approach, dubbed the ‘xedni calculus’, 
for attacking the ECDLP. The xedni idea ‘turns the index calculus on its head’ 
by first lifting a bounded number (nine) of points to Q then finding a lift £ /Q 
of E to fit the lifted points. This approach circumvents the difficulty of lifting 
points and does not require the lift £ for E to have a large rank. In fact the 
success of this method depends on the lifted points being linearly dependent in 
F(Q). The probability for this to occur would presumably be low. To increase the 
probability Silverman imposed additional conditions on the lift based on some 
heuristic arguments involving the Birch-Swinnerton-Dyer Conjecture. However, 
a subsequent analysis by Silverman and Jacobson et al [2] shows that with the 
xedni algorithm the probability of success in finding a discrete logarithm on an 
elliptic curve over a finite field is in fact negligible asymptotically speaking. 

The results in this paper strengthen the observations of Miller [6] and the 
analysis of Silverman et al [2,10] by deriving necessary but difficult-to-satisfy 
conditions for any index-calculus type of method which involves the lifting idea 
to solve the ECDLP in subexponential time. 

The center piece of our analysis is the following result concerning lifting an 
elliptic curve over a finite fields together with a finite set of points. Let E be an 
elliptic curve over a finite field Fp. For r G Z>o and h G M>o> let riEir, h) denote 
the number of (r -|- l)-tuples A = (Pqj ■■■) Pr) with Pi in some cyclic subgroup of 
E{¥p) so that (E, A) can be lifted to some {£\, A) over Q with the rank of f (Q) 
bounded by r and the canonical heights of the points in A bounded by h. We 
show that nE{r, h) is bounded by 2‘^(’’^)(li/log where N = |if(Fp)|. 

From the theorem we deduce the following conclusions. 

With the approach such as the index calculus method, where one lifts an ellip- 
tic curve E /Fp to an elliptic curve £ /Q before lifting random points (generated 
from the two points in question), in order to possibly achieve subexponential 
running time (such as 0(exp(c(logp)^/^(loglogp)^/^)), the rank of £ needs to 
grow at least as fast as (logp)^/^ as p grows. 

With the approach such as the xedni calculus method, where one lifts a set 
of random points (generated from the two points in question) then constructs a 
curve £ to fit the lifted points, in order to possibly achieve subexponential run- 
ning time (such as 0(exp(c(logp)^/^(loglogp)^/^)), the number of lifted points 
needs to grow at least as fast as {logp^P as p grows. To underscore the difficulty 
in meeting this condition, we show that a method for lifting an asymptotically 
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increasing number (such as (logp)^/^) of random points on an elliptic curve over 
Fp to rational points of canonical height bounded subexponential in logp on an 
elliptic curve over Q implies a method for constructing elliptic curves over Q of 
arbitrarily large rank. On the other hand, bounding the number of lifted points, 
as the xedni algorithm in [8], results in asymptotically negligible probability of 
success in solving the ECDLP. 

Our analysis depends on a conjecture of Lang [4] that the canonical height of 
any nonzero rational point on an elliptic curve £ over Q is bounded from below 
by clog |L\(£i)| where c is a universal constant independent of £ and A{£) is the 
minimal discriminant of £. Lang’s conjecture is the only unproven assumption 
needed throughout this paper. (The results in [2,10] depend on Lang’s conjecture 
as well as other heuristic assumptions.) It is worth mentioning that the conjecture 
has been proven to a large extent [1,9]. 

It should be pointed out that our results are asymptotic in nature and 
they leave open the possibility for the index-calculus idea (including the xedni 
method) to successfully attack the ECDLP in the lower range of p. 

The rest of this paper is organized as follows. In Section 2 we prove the 
theorem concerning the lifting problem and in Section 3 we relate the result to 
the elliptic curve discrete logarithm problem. 

2 The Lifting Problem 

Let E be an elliptic curve over a finite field Fp and A = (Pi, ..., Pm) with 

Pi G P(Fp). Let £ be an elliptic curve over Q and A = (Pi, ..., Vm) with 

Pi G i£(Q). We say that (P, A) is lifted to (£,A) if E can be obtained as the 
reduction of £ modulo p with Pi as the reduction of Pi modulo p for i = 1, ..., 
m. We say that A is lifted with E with canonical height bounded by h if the 
canonical height of Pi is bounded by h for z = 1, ..., m. 

Let h{V) denote the canonical height [7] of P for P G £1(Q). Let 

IV(£:,6) = #{Pg£(Q):MP)<6}. 

Let r = r{£) be the rank of i£(Q), T be the number of torsion points in £(Q), 

and R be the regulator of £ over Q. Then it is known [4] that 

/ b 

N{£,b)-^Tar[^j^j , 

where ar is the volume of the unit r-ball. We assume Lang’s conjecture [4] that 

^(P) >clog|Z\(£:)| 

for some constant c independent of £, where A{£) denotes the minimal discrim- 
inant of £. Then from 




pi/’' > 



min h{V) 




380 Ming-Deh A. Huang, Ka Lam Kueh, and Ki-Seng Tan 



where the minimum is over all nonzero V G i£(Q) (see [4]), and 

1 /27reV/^ 

7^ V“ ) ’ 

and that T < 16 (see [5]), it follows that for r > 1 
for some positive constant ci independent of £. 

Proposition 1. There exists a positive constant c such that for all elliptic curves 
E defined over Q, if the rank of £{Q) is no greater than r, then for any Vq, 

Vr in £{Q) with h{Vi) < h, there exist integers Ci with |cj| < 2°’’ 
such that '^iCi'Pi = 0, where A is the minimal discriminant of £. 

Proof: For V G 5(Q), let IIPH = \Jh{V). For Oi G {0, m— 1}, 

r r r 

II ^ aiViW < ^ |ai|||Pi|| < Vh'^ \ai\ < m{r + l)Vh. 



So 

r 

h(^aiVi) < wf^{r+ Vfh. 

i=0 

Since the number of (oq, ■■■,ar) with Oi G {0, 1} is if 

N{£,rn^{r + \Yh) < , (2) 

then there must exist two distinct (oq, a^) and (6o, hr) with Oj, bi G {0, . . 
m — 1} such that OiVi = biVi, and hence CiVi = 0 with Ci = Oi — bi, 
so |ci| < m. From Eq. (1), 



N{£,m^{r+lfh) < 



/ mf{r+ l)‘^h 

V log|Z\| 



r/2 



for some constant ci independent of £. Hence (2) holds if m > 2'^’’ (ft,/ log |Z\|)’’/^ 
where c is a constant independent of £. 



Theorem 1. Let E be an elliptic curve over a finite field Fp. For r G Z>o and 
ft G M>o> let nE{r, ft) denote the number of X = {Pq, Pr) with Pi in some cyclic 
subgroup of E{¥p) so that {E, A) can be lifted to some {£, A) over Q with the 
canonical heights of the points in A bounded by ft and the rank of £{Q) bounded 
by r. Then nE(r,h) is bounded by 2‘^(’’^)(ft/log |Z\|)‘^^’’^^fV’’ where N = |E(Fp)| 
and A is the minimal discriminant of £. . 
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Proof: Let A = {Po,...,Pr) with Pi in some cyclic subgroup of if(Fp) with 
a generator S. Suppose {E,X) is lifted to some (£,A) with canonical height 
bounded by h. Suppose A = {Vq, ...,Vr)- If the rank of F(Q) is bounded by r, 
then from Proposition 1 it follows that there exist integers Cj such that 

CiVi = 0 , 



where 




and A is the minimal discriminant of £. 
Suppose Pi = rriiS. Then 



r/2 



0 = CiPi = (yy Cimi)S. 

i i 

yy CiTTii = 0 (mod N) 



I 

where N is the order of S. Now n£;(r, h) is bounded by the number of (mo, rrir) 
such that CiWii = 0 (mod N) and \ci\ is bounded by 



M = 2“’'' 



h 

log|Z\| 



r/2 



For each c = (cq, ■■,Cr), let ric denote the number of (mo, ...,mr) mod N such 
that 

cquio + ... + CrUir = 0 (mod N). 

Suppose the g.c.d. of co, ..., Cr is g, then 



Uc < gN'^ < MN'^. 



So 

riE{r, h) < {2M = 2‘^(’'')(/i/log 

3 Analysis on the Index-Calcnlus Approach to ECDLP 

In the elliptic curve discrete logarithm problem we are given an elliptic curve E 
over a finite field Fp, and two points S,T G E{¥p), and the problem is to find 
an integer m (if it exists) so that mS = T. 

A natural generalization of the index calculus method for the ECDLP can 
be outlined as follows. 

1. Find an elliptic curve £ defined over Q whose reduction mod pis E. Suppose 
£(Q) has rank r with a basis Vi, i = I, ..., r, and suppose Pi G E(¥p) is the 
reduction of Vi mod p. 
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2. For random integer j, lift jS to some S' G i£(Q), and write S' in terms of 
Vi (up to a torsion point). Each S' yields a linear relation on the discrete 
logarithms of Pi. With r many linearly independent relations we can solve 
for the discrete logarithms logg(Pi) of all Pi. 

3. For random integer j, lift T + jS to some S' G f (Q), and write S' in terms 
of Vi. Then log 5 (T) can be determined. 



For the method to work in subexponential time, r + 1 random points in 
E(Fp) need to be lifted to points in F(Q) of canonical height bounded by some h 
which can be at most subexponential in logp. The number of such (r + 1)- 
tuples of points in E(Fp) cannot be greater than nE(r,h), which by Theo- 
rem 1 is bounded by \og\A\)^^'' '>N''. Hence the success probability 



is bounded by 



20G^)(;i/log|zi|)0(’' 
N 



Since N can be in the order of p, for the 



success probability to be at least l/exp[(logp)^/^(loglogp)^/^], say, it is nec- 
essary that log ft- > c'logp for some constant c' . Since ft can be at most 
ea;p[0(l)(logp)^/^(loglogp)^/^], the number of lifted points r-|- 1 needs to be at 
least in the order of (logp)^/^ as p grows. 

The same conclusion can also be deduced from Proposition 1. Let ft be an 
upper bound on h{Vi) and h{S') where S' lifts a point P in E(Fp). Then from 
Proposition 1 it follows that there exist integers Ci with absolute values bounded 
by 2‘^’’^(ft/log \ A\y/‘^ such that 



CqS' CiV I ... -f CrV r — 0 



SO 

CqP C\Pi -\- ... -\- CrPr = 0. 
The number of P G P(Fp) satisfying 



CqP C\Pi -\- ... -\- CrPr — 0 



with \a\ < 2=’'"(ft/log|Z\|)’'/2 is bounded by 20 P")ftO(P). 

It follows that the probability that a random P can be lifted to some S' with 



height bounded by ft is no greater than 



20(r^)^0(.^) 



. For the probability to be 



at least l/ea;p[(logp)^/^(loglogp)^/^], say, it is necessary that r^logft > c'logp 
for some constant c'. Even if we allow the points to be lifted to subexponential 
canonical height so that ft is about ea;p[(logp)^/^(loglogp)^/^], the rank r of S 
still needs to be at least in the order of (logp)^/^. 

Note that the observation above holds regardless of the method used to con- 
struct £ and lift a point from E to £. The fact that the rank of £ needs to grow 
at least as fast as (logp)^/^ as p grows already poses a significant difficulty for 
the index calculus method to work. 

Next we turn our attention to the xedni calculus method for the elliptic curve 
discrete logarithm problem. Below is a general outline for the method. 
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1. Generate random Pq, Pr with Pi = atS + biT where Oj, bt are random 
integers. 

2. Lift Pi to some Vi over Q, then construct an elliptic curve S over Q so that 
the pair S and {Vo, ■■■,Vr) is a lift of E and {Po, ■■■,Pr)- 

3. If the rank of £(Q) is no greater than r, then Vo, ■■■, Vr are integrally 
dependent, so that 

^ c^Vi = 0 

i 

for some integers Cj, then upon reduction mod p we have 

0 = ^ Ci{aiS + biT) = Ciai)S + Cibi)T. 
i i i 

From this the discrete logarithm of T in terms of S can be obtained with 
high probability, since Oi and bi are randomly chosen. 

The xedni algorithm of Silverman is consistent with the outline above, with 
r set at 9, and as mentioned before, additional conditions imposed on £. 

For the method to work in subexponential time, r + 1 random points in 
if(Fp) need to be lifted to points of canonical height at most subexponential 
in logp on some £ over Q of rank at most r. The number of random (r + 1)- 
tuples A = {Po,...,Pr) is bounded by For a A to lead to a success in 

finding the discrete logarithm we need {E,\) to be lifted to some {£,A) over 
Q with the canonical heights of the points in A bounded by h and the rank of 
£(Q) bounded by r. The number of such (r + l)-tuples cannot be greater than 
nE{r,h), which by Theorem 1 is bounded by N'^ . Hence 

the success probability is bounded by ^ Since N can be in the 

order of p, for the success probability to be at least l/exp[(logp)^/^(loglogp)^/^], 
say, it is necessary that log h> c' logp for some constant d . Since h is at most 
ea;p[0(l)(logp)^/^(loglogp)^/^], the number of lifted points r + 1 needs to be 
at least in the order of (logp)^/^ as p grows. This is true regardless of how the 
curve £ is constructed for each (r + l)-tuple of points in if(Fp). In particular, 
for bounded r (such as the case with the xedni method in [8]), the probability 
of success tends to zero asymptotically with p. Hence the xedni calculus method 
as described in [8] cannot work as a subexponential algorithm asymptotically. 

To extend the scope of applicability of the xedni calculus idea, we would need 
to increase the the number of random points on an elliptic curve to be lifted to 
rational points of reasonably bounded canonical height on an elliptic curve over 
Q. But the difficulty of such task is underscored by that of constructing elliptic 
curves of large rank over Q as reasoned below. 

Let m_E(r, h) denote the number of A = {Po, ..., Pr) with Pi in E(¥p) so that 
{E, A) can be lifted to some {£, A) over Q with the canonical heights of the 
points in A bounded by h. For any fixed r, suppose for elliptic curves E over 
Fp, mE{r,h) > where where c is a positive constant less than 1 and 

log ft-/ log p tends to 0 as p tends to infinity (say ft is subexponential in logp). 
Then by Theorem 1, m£;(r, ft) > ri£;(r, ft) for sufficiently large p and E with 
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cyclic group iil(Fp). It follows that for sufficiently large p, some elliptic curve 
over Q lifting some elliptic curve E over Fp together with some r-tuple of points 
on E must have rank at least r. 
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Abstract. In this paper, we propose a three participants variation of 
the DifBe-Hellman protocol. This variation is based on the Weil and 
Tate pairings on elliptic curves, which were first used in cryptography as 
cryptanalytic tools for reducing the discrete logarithm problem on some 
elliptic curves to the discrete logarithm problem in a finite field. 



1 Introduction 

Since its discovery in 1976, the Diffie-Hellman protocol has become one of the 
most famous and largely used cryptographic primitive. In its basic version, it is 
an efficient solution to the problem of creating a common secret between two 
participants. Since this protocol is also used as a building block in many complex 
cryptographic protocols, finding a generalization of Diffie-Hellman would give a 
new tool and might lead to new and more efficient protocols. 

In this paper, we show that the Weil and Tate pairings can be used to build 
a tripartite generalization of the Diffie-Hellman protocol. These pairings were 
first used in cryptography as cryptanalytic tools to reduce the complexity of the 
discrete logarithm problem on some “weak” elliptic curves. Of course, the prob- 
lem of setting a common key between more than two participants has already 
been addressed (see the protocol for conference keying in [1]). However, all the 
known techniques require at least two round of communication. In some proto- 
cols having these two rounds can be somewhat cumbersome, and a single round 
would be much preferable. To give an example, exchanging an email message 
key with a two round Diffie-Hellman protocol would require both participants 
to be connected at the same time, which is a very undesirable property for a 
key exchange protocol. For this reason, we believe that the one round tripartite 
Diffie-Hellman presented here is a real improvement over conference keying even 
though the computational cost will be somewhat higher. 

2 The Discrete Logarithm Problem on Weak Elliptic 
Curve 

The discrete logarithm problem on elliptic curves is now playing an increasingly 
important role in cryptography. When elliptic curve cryptosystems where first 



W. Bosnia (Ed.): ANTS-IV, LNCS 1838, pp. 385-393, 2000. 
© Springer- Verlag Berlin Heidelberg 2000 




386 Antoine Joux 



proposed in [9] , computing the number of points of a given curve was a challeng- 
ing task, since the Schoof, Elkies and Atkin algorithm was not yet mature (for a 
survey of this algorithm see [6]). For this reason and also to simplify the addition 
formulas, the idea of using special curves quickly arose. However, it was shown 
later on that some of these special cases are not good enough. Today, three weak 
special cases have been identified. In one of them, the discrete logarithm problem 
becomes easy (i.e. polynomial time) as was shown in [11,10]. This easiest case 
happens when the number of points of the elliptic curve over Fp is exactly p. 
In the two other cases, the discrete logarithm problem on the elliptic curve is 
transformed into a discrete logarithm problem in a small extension of the field 
of definition of the elliptic curve. These two reductions are called the Menezes, 
Okamoto, Vanstone (MOV) reduction [8] and the Frey, Riick (FR) reduction 
[3]. A survey of these reductions was published at Eurocrypt’99 [4], and gave a 
comparison of these two reductions. The conclusion was the FR reduction can 
be applied to more curves than the MOV reduction and moreover that it can be 
computed faster than the MOV reduction. Thus for all practical usage, the au- 
thors recommend the FR reduction. However, they claim that the computation 
of the FR and MOV reduction may be a heavy load. We will show that in fact 
this is not the case and that these reductions can be turned from cryptanalytic 
to cryptographic tools. 

Pairings on Elliptic Curve 

The MOV and FR reductions are both based on a bilinear pairing, in the MOV 
case it is the Weil pairing and in the FR case it is (a variation of) the Tate 
pairing. In the sequel, we describe these pairings for an elliptic curve E defined 
over Fp. In order to define these pairings, we first need to introduce the function 
field and the divisors of the elliptic curve. Very informally, the function field 
K{E) of E is the set of rational map in x and y modulo the equation of E (e.g. 

— ax — b). A divisor D is an element of the free group generated by 
the points on E, i.e. it can be written as a finite formal sum: D = 
where the Pi are points on E and the are integers. In the sequel, we will only 
consider divisors of degree 0, i.e. such that ^ ■ Oi = 0. 

Given any function / in K(E), we can build a degree 0 divisor div{f) from 
the zeros and poles of / simply by forming the formal sum of the zeroes (with 
multiplicity) minus the formal sum of the poles (with multiplicity) . Any divisor 
D = div{f) will be called a principal divisor. In the reverse direction, testing 
whether a degree 0 divisor D = ’^iai(Pi) is principal or not, can be done by 
evaluating at Pi on E. The result will be the point at infinity if and only if D 
is principal. 

Given a function / in K{E) and a point P oi E, f can be evaluated at P by 
substituting the coordinates of P for x and y in any rational map representing 
/. The function / can also be evaluates at a divisor D = ^iai(Pi), using the 
following definition: 

i 
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Using these notions, we can now define the Weil pairing: it is a bilinear 
function from the torsion group E[n] to the multiplicative group /i„ of n-th 
roots of unity in some extension of Fp, say Fpfc . Given two n-torsion points P 
and Q, we compute their pairing e„(P, Q) by finding two functions fp and /q 
such that div{fp) = n{P) — n{0) and div{fQ) = n{Q) — n{0), and by evaluating: 

en{P,Q) = fp{Q)/fQ{P). 

This pairing e„ : E[n] x E[n] — > /i„ is bilinear and non-degenerate. This 
means that en{aP,bQ) = e„(P, and that for some values of P and Q, we 
have e„(P, Q) yf 1. We can easily see that given a point X “independent” from 
P and Q, we can reduce the discrete logarithm problem Q = XP on the elliptic 
curve to the discrete logarithm problem e„(Q, AT) = e„(P, in Fpfc . 

The variant of the Tate pairing described in [3] is more complicated, since 
it operates on divisors instead of points. The Tate pairing operates on n-fold 
divisors, i.e. divisors D such that nD is principal, it takes values in and it is 
bilinear and non-degenerate. Given two n-fold divisors D\ and defined over 
an extension Fpj, that contains the n-th roots of unity, we find fp^ and fp^ such 
that div{fp-f) = nDi and div{fp^) = nD 2 - The Tate pairing of Di and D 2 is 
then defined as: 

This pairing is also bilinear and non-degenerate. Moreover, for the purpose 
of discrete logarithm reduction, the Tate pairing t„(Di, £> 2 ) can easily be trans- 
formed into a pairing that involves points. One can simply fix two points R and 
5, and remark that t„((AP) - (O), (R) - (5)) = t„((P) - (O), (R) - (S))^. 

For more details about the properties and definitions of the Weil and Tate 
pairing, we refer the reader to [8,3,4]. 



3 A Tripartite DifRe— Heilman Protocol 

In this section, we want to build an analog of the Diffie-Hellman protocol, that 
involves three participants A, B and C , requires a single pass of communica- 
tions and allows the construction of a common secret Ka,b,c- By a single pass 
of communication, we mean that each participant is allowed to talk once and 
broadcast some data to the other two. The main idea is as in ordinary Diffie- 
Hellman, we start from some elliptic curve E and some point P. Then A, B 
and C each chose a random number (a, b or c) and they respectively compute 
Pa = aP, Pp = bP and Pc = cP and broadcast these values. Then they respec- 
tively compute P{a, Pp, Pc), E{b, Pa, Pc) and P{c, Pa, Pb), where the function 
P is chosen in a way that ensures that these numbers will be equal and that 
this common value Ka,b,c will be hard to compute given Pa, Pp and Pc- The 
problem now is to find such an P . 

Using the Weil pairing, it is seems very easy to define such an P using the 
following formula: 



Pw{x,P,Q) = en{P,Q)^. 
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With this definition, one can easily check that: 



Fw{a, Pbi Pc) = Fw{h^ Pa, Pc) = Fw{c, Pa, Pb) = Fw{^, P, 

However, this function is not satisfying because e„(P, P) = 1 and thus Ka,b,c 
is a constant. Nevertheless, the basic idea is quite sound and can in fact be 
implemented if we use two independent points P, and Q and if we have the 
three participants compute and broadcast {Pa,Qa), {Pb,Qb) and (Pc,Qc)- 
Then A, B and C can respectively compute Fw{a, Pb,Qc) = Fw{a,QB, Pc), 
Fw{b, Pa,Qc) = Fw{b,QA, Pc) and Fw{c, Pa,Qb) = Fw{c,Qa,Pb)- More- 
over, all these values are equal and thanks to the independence of P and Q, they 
are not constant. 

Moreover, using two points P and Q, it is easy to use the Tate pairing instead 
of the Weil pairing, and to define another function F as: 



Ft{x,Di,D2) = UDi,D2)F 

Then A, B and C can respectively compute: 

FT{a,{PB) - {Qb),{Pc + Qc) - {O)) = 

Fria, (Pc) - (Qc), {Pb + Qb) ~ {O)), 

Frib, {Pa) ~ {Qa), {Pc + Qc) ~ {O)) = 

Fr{b, {Pc) - {Qc), {Pa + Qa) ~ {O)), 

Ft{c, {Pb) - {Qb), {Pa + Qa) ~ (O)) = 

Ft{c, {Pa) - {Qa), {Pb + Qb) ~ {O)). 

Because of the bilinearity of the pairing, all these numbers are equal and because 
of the non-degeneracy, their common value 

Ft{1,{P)-{Q),{P+Q)-{0))-»^ 

is not independent from the choice of a, b and c. 

Since Ft is based on the Tate pairing, it will be faster to evaluate then Fw 
(see the general remark about the efficiency of the Tate pairing versus that of 
the Weil pairing in [4]). Finally, our tripartite Diffie-Hellman protocol can be 
summarized as follows: 



Alice 


Bob 


Charlie 


Choose a 

Compute {Pa,Qa) 


Choose b 

Compute {Pb,Qb) 


Choose c 

Compute {Pc,Qc) 


Broadcast Pa, Pb, Pc and Qa, Qb, Qc- 


Compute the common key as: 

Fr{a, {Pb) ~ {Qb), {Pc + Qc) ~ {O)) 

Fr{b, {Pa) ~ {Qa), {Pc + Qc) ~ {O)) 

Ft{c, {Pb) - {Qb), {Pa + Qa) ~ (0)) 
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Choice of Parameters and Construction of the Elliptic Curve 

For the tripartite DifRe-Hellman protocol to be efficient, we need to choose 
elliptic curves such that the pairing can be efficiently computed. This means 
that the group /i„ should be in a small extension Fp, i.e. k should be small. 
Moreover, we need to choose two points P and Q such that the pairing will be 
non-degenerate, this point can easily be checked by testing whether e„(P, Q) or 
tn{{P) — (Q), {P + Q) — (O)) is 1 or not. Note that when fc 1 at least one of 
the points P and Q must be defined over the extension Fpfc rather than over Fp 
otherwise the pairing will always be degenerate. 

Two kind of curves are very promising for this tripartite Diffie-Hellman: 
supersingular curves (which leads to fc = 2 according to the MOV reduction), 
and curves of trace 2 (which leads to fc = 1 according to the FR reduction). It 
might seem strange to use elliptic curves which are known to be weaker than 
random curves, however, since we are also mixing in exponentiation in Fpfc , we 
need to choose a large enough p for the discrete logarithm in Fpfc to be hard and 
then nobody knows how to compute discrete logarithms on the elliptic curve. 
The first kind of curve, i.e. supersingular curves, is well known and very easy to 
build. However, curves of trace 2 are not so easy to construct, in fact, we only 
known how to construct such curve when p — 1 is a square or a small multiple 
of a square (see [5] or for some examples [4]). This is a pity because curves of 
trace 2 with a squarefree p — 1 would allow us to work with a single point over 
Fp instead of two which would be very nice and efficient. 



4 Efficient Implementation of the Pairing 

The main step when computing the Weil or the Tate pairing is given a n-fold 
divisor D = (V) — (V), to write the principal divisor nD as the divisor of a 
(bivariate) function / denoted by div{f). Then we need to evaluate / at some 
other point Z. There exists a standard method to do that, which is based on the 
fact that every divisor can be written as (P) — (O) -I- div{f) for some point P 
and some function /, and that adding two divisors of that form is easy. Indeed, 
if 

D={P)-{0) + divif), 

D' = {P') - (O) + div{f) 

then 

D + D' = {P + P')~ (O) + diviffg), 

where g = l/v with I the line through P and P' and v the vertical line through 
P+P'. 

As explained in [7], when writing nD as div{f), f cannot be expressed as 
an expanded polynomial (which would be exponentially large) but should be 
kept in factored form. However, even in factored form, writing down / is quite 
costly. As an example, the data in [4] shows that such a computation took about 
40000 seconds for a supersingular curve when using a 50-digit prime p. This is 
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not acceptable since with supersingular curves we want to work with a prime 
number of at least 100 or 150 digits. 

In fact, a much better approach is to avoid the computation of / and to 
directly compute f{Z). This is easily done by keeping for each intermediate 
divisor D the values of P and f{Z) and by forgetting /. Computing ffg{Z) is 
easily performed by multiplying f{Z), f'{Z) and g{Z). Thus at each step, we 
only need to evaluate two linear polynomials, to compute one inverse and to 
multiply a couple of numbers. Using this approach and the ZEN library [2], we 
see in the following example that the Tate pairing can be computed in a single 
second on a Pentium 11-400 processor for a supersingular curve defined over a 
prime field of more than 150 digits. 



A Small Example 



In this section, we give an example of the tripartite Diffie-Hellman using a 
supersingular curve. We chose a prime p of more than 512 bits: 

p = 48267777815770043535044410856360047038953960729113574 
29530850774144832990078179684573230519991072031530329 
37333023591271636050696817523671646492380723773419011. 



We are working on the supersingular curve defined by -I- x. Since we 

need to work in an extension field with p^ elements, we define this field from the 
irreducible polynomial -I- 1, and we denote the square root of 1 by the letter 
i. 

Remark that p was chosen in such a way that the large (160 bits) prime q 
divides p+ 1, where: 

q = 593917583375891588584754753148372137203682206097. 



We then choose our two points P and Q as points of order q: 



P = (4419030020021957060597995505214357695235725551511568 
68511701918183168420954869076254808843953176168634019 
27551006066189692708095924815897927498508535823262371, 
26090947680860922395540330613428690525406329616428470 
73807303133884126088547738030713042022034220476530186 
5163480203757570223664606235381540801075563801118751) 
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Q = (4174183901517981791573276838146590144608495183505084 
36411447781417311430237331232958577456865429161040089 
806217226455983348248260335272068783343983410685645620, 
85984079438328066829535503806402848425113755688042614 
53460943539888201506845050435386547281506353153165721 
0019063972911218641810155964304683033635085838106425z) 

Using the Tate pairing we can compute 

Ft{1,{P)-{Q),{P+Q)-{0)) = 

321226044133092484635656769053049333393058975135298190055 

149195187870368117448022160010655718390434221411264718401 

205796045961343192326955779028644235767724655Z+ 

188248671808397625173631034231316372667592199772896982055 

003439080715924660694288538218628657757570098468723289223 

254974186814834824668646542592184808038517084 

Then for a = 4, 6 = 7 and c = 28 we compute 

Fr(a, (bP) - (bQ), {cP + cQ) - (O)) = 

21704655273258595020185058036714661585432952223857344835 

67773957210551020200586870416066057916675619991969502192 

64185045830782800156145170386696601496318727119Z+ 

18547967545356005000241995328735966990113791703635028416 

23483761786522135284562773843989027568976094155038271048 

94436481787700370161453899874562738321254026146 

and we check that indeed 



Fria, (bP) - (bQ), (cP + cQ) - (O)) = Ft(1, (P) - (Q), (P+Q)~ (0))“'’" 

Each evaluation of Ft took 1 second on a Pentium 11-400 PC running under 
linux, which is very efficient compared to the 40000 seconds (on a Pentium-75) 
in [4]. 

5 Security Issues 

Clearly in order to be secure the tripartite Diffie-Hellman described here requires 
the discrete logarithm on the chosen elliptic curve to be hard, and the discrete 
logarithm in the finite field Fpfc to be hard. Since we placed ourselves in the cases 
where either the MOV or the FR reduction applies, the hardness of the elliptic 
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curve discrete log implies the hardness of the finite field discrete log and we can 
remove the second condition. This is a simple restatement of the fact that when 
the finite field discrete log, then to solve the elliptic curve discrete log we simply 
transport the problem in the finite field using the pairing and then solve the 
problem in the finite field. However, it is not known whether the elliptic curve 
discrete logarithm on a weak curve is as hard as the discrete logarithm in the 
corresponding finite field (in the sense of the MOV or FR reduction). In fact, 
this is a very interesting open problem. Moreover, as in the Diffie-Hellman case 
this is not the whole story, some Diffie-Hellman like problem and Diffie-Hellman 
like decision problem should be hard in order to get security. 

Quite amusingly, we should note that on curves where either the MOV or FR 
reduction applies, the usual Diffie-Hellman decision problem is mostly easy. Re- 
member that the usual Diffie-Hellman problem is given a quadruple (5, 5“, 5^, 5'^) 
to decide whether c = ab. This problem can also be expressed with the following 
formulation which is slightly different. Given a quadruple (g,g°‘,h,h^), decide 
whether a = b. Now on an elliptic curve where the MOV reduction applies, we 
can easily test for a quadruple (P, aP, Q, bQ) whether a = 5, it suffices to com- 
pute en{aP,Q) and e„(P, 6(3) and to compare them. This test works as soon 
as P and Q are independent (i.e. when e„(P, (3) yf 1). Of course, in the FR 
case, such a test also exists. More precisely, one can test for the equality of 
tn{{aP) — (O), (XQ) — (Q)) and tn((P) — (O), (XbQ) — (bQ)), where A is essen- 
tially any constant number (some values of A are excluded, for example A = 1 
is not allowed). Note than when P and Q are not independent, the test usually 
doesn’t work, thus some cases of the usual Diffie-Hellman decision problem are 
still hard on these elliptic curves. 

With the current knowledge of elliptic curves, we believe that this system 
is secure in practice as soon as the discrete logarithm in Fpfc is hard. For the 
supersingular case {k = 2), we think that p should be a 512 bits prime. In the 
trace 2 case {k = 1), we recommend to choose a 1024 bits prime. Moreover, 
the usual precautions should be taken, i.e. some large prime q should divide the 
order of the elliptic curve, all the points involved in the computation should be 
of order q, and we should use the pairing Cq or tq. 



6 Conclusion 



In this article, we described a generalization of the Diffie-Hellman protocol to 
three parties using the Weil or Tate pairing on elliptic curves. We also showed 
that this pairing can be implemented much more efficiently than previously 
shown in [4] . Therefore, this new protocol seems quite promising as a new build- 
ing block to construct new and efficient complex cryptographic protocols. On the 
other hand, we sincerely hope that people will try to attack it, since finding a 
weakness in this protocol would certainly give some new insight in the difficulty 
of the discrete logarithm on elliptic curves. 
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Abstract. In the paper an upper bound is established for certain expo- 
nential sums, analogous to Gaussian sums, defined on the points of an 
elliptic curve over a prime finite field. The bound is applied to prove the 
existence of group generators for the set of points on an elliptic curve 
over Fq among certain sets of bounded size. We apply this estimate to 
obtain a deterministic algorithm for finding generators of the 

group in echelon form, and in particular to determine its group structure. 



1 Introduction and Notations 

Let q = he & prime power and let S be an elliptic curve over a finite field 
of q elements given by a Weierstrass equation 

y'^ + {aix + a3)y = + a2x'^ + a4X + ae- ( 1 ) 

The set F(Fq) of points over F^, together with the point O at infinity as identity, 
forms an Abelian group. The cardinality of F(Fq) is N, where 

|V - g - 1| < 

Moreover, as a group, F(Fq) is isomorphic to Z/M x Z/L for unique integers M 
and L with L \ M and N = ML. The number M is called the exponent of F(Fq). 
Points P and Q in F(Fq) are said to be echelonized generators if the order of P 
is M, the order of Q is L, and any point in F(Fq) can be written in the form 
mP + £Q with 1 < m < M and 1 < ^ < L. 

Although there exists a deterministic polynomial time algorithm to find the 
number of Fg-rational points N due to R. Schoof [11] (see also [4,5,16] for ref- 
erences to further theoretical and practical improvements of this algorithm), 
finding the group structure, or equivalently the exponent M, appears to be a 
much harder problem. 

Once the group order N and the factorization of r = gcd{q — 1,N) are 
known, there exists a probabilistic algorithm to compute the group structure in 
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expected polynomial time (see [9,10]). We note that the existence of the Weil 
pairing (see [18]) implies that L divides r. Thus, using the factorization of r and 
the nondegeneracy of the Weil pairing, the algorithm finds the value of L. The 
best possible bound on r is + 1, but for a random curve the value of r tends 
to be small, in which case the algorithm is efficient. 

We now describe the exponential sums which are the subject of study in this 
work. Let P and Q be echelonized generators for f (F^). For a real number z or 
element of Z/nZ, we define 



e„(z) = exp(27Tzz/n). 

The group 17 = Hom(F(Fg), C*) of characters on F(Fg) can be described by the 
set: 



17 = {w I oj{mP + IQ') = 6m (am) (bi) for 0 < a < M, 0 <b < L}. 

Similarly the group = Hom(Fg,C*) of additive characters on F^ can be de- 
scribed by the set: 



<P = {ipl = Bp (Tr(az)) for a G F^}, 

where Tr(a;) is the trace of a; G F^ to Fp (see Chapter 2 of [8]). The identity 
elements of the groups 17 and are called trivial characters. 

Let Fq(F) be the function field of the curve F. It is generated by the functions 
X and y, satisfying the Weierstrass equation (1) of the curve, and such that 
P = (x{P),y{P)) for each P G F(Fg) — {O}. 

For characters w G 17 and z/l G >F, and a function / G Fg(F), we define the 
sum 

Pe£{¥^) 

f(P)¥^'=o 

In this work we estimate the exponential sums S{uj, ip, /). In particular we will 
be interested in the sums for f = x or f = y. The bounds obtained generalize 
and improve previous bounds from [13,14]. We apply this bound to design a 
deterministic algorithm to compute the group structure of F(Fg) and to find 
echelonized generators in time 

In the next section we recall some classical results on L-functions of curves, 
and relate these to S{oj,ip, /). 

Throughout the paper log z denotes the natural logarithm of z. 

2 X-Functions of Curves 

Let C be an irreducible projective curve over F^ of genus g. The divisor group 
is the free abelian group of formal sums of prime places of Fg(C). For a fixed 
algebraic closure F^ of F^ we can identify a prime place with a Galois orbit 
{Pi, . . . , Pd} of points in C(Fg), and define d = deg(*P) to be its degree. 
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A character % of the divisor group of Fg(C) is a map to C, with image in 
a finite set {0} U e„(Z) and which is a homomorphism to C* on divisors with 
support outside of a finite set of prime places. Associated to x is a cyclic Galois 
cover 7T : A ^ C and a divisor f(x) called the conductor, such that tt is unramified 
outside of the support of f(x). 

We define the following character sums 

<^m{x)= deg(«P)x(*P), 

deg^<m 



taken over all prime places of Fg(C) of degree deg*P < m. We define an L- 
function 

L{C,t,x) = exp cr™(x)t'"/mj , 
where exp : — > C[[t]] is given by 



.w h(tT 

exp{h{t)) = . 

n\ 

n—O 

The following proposition for L{C,t,x) appears as Theorem A of [2] or Theorem 6 
of Chapter 7 of [20] . 

Proposition 1. L{C,t,x) is a polynomial of degree 

D = 2g-2 + degf(x) 

where f(x) is the conductor of x- If X is a product of two characters xi X2 
which are ramified in disjoint sets of divisors then 

degf(x) = degf(xi) + degf(x2)- 

We remark the second statement is applicable in particular if one of charac- 
ters is totally unramified. 

We next recall the statement of the Riemann Hypothesis for function fields. 
Proposition 2. Let di, . . . , 1?^ be zeros of L{C, t, x) in C. Then 

am{x) = -{IiT + ■■■ + !) d), 

and each zero satisfies jdij = 
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3 Exponential Sums on Elliptic Curves 

We recall the following standard lemma on character groups of abelian groups. 

Lemma 1. Let G he an abelian group and let G = Hom(G, C*) he its dual group. 
Then for any element \ of G, we have 

where \o ^ G is the trivial character. 

In particular, we apply the bound to the pairs {•?■, F,} and {F(Fq), 17}. By 
the canonical isomorphism of G with the dual of G, the lemma is symmetrical 
in G and G. 

As an immediate application of Lemma 1 we observe that if fjo is the trivial 
character, then 



*/x = Xo, 
*/x Xo, 



Pe£(F,) Pe£{¥^) 

f(P)^oo /(P)=oo 

Thus we see that the interesting part of the exponential sum comes from the 
character ipo f, which defines an Artin-Schreier extension of Fg(£), as studied in 
Bombieri [2, Section VI]. We also remark that the exponential sums S{oJo, ip, /) 
with the trivial character ojq € G have been estimated in [2] . 

Let / be a nonconstant function on £. We write the divisor of poles of / as 

i 

(/)oo = 



where, in particular, 



t 

deg(/) = nideg{%). (2) 

i=l 

In particular, degf = 2 if / = x, and degf = 3 if / = y. With this notation we 
have the following theorem. 

Theorem 1. The character to determines an unramified character, and ip o f 
determines a character of conductor X)i=i where m, < rii + 1 with equality 

if and only if{ni,q) = 1. The exponential sum satisfies the bound 

t 

\S{uj,ip,f)\ < ^mideg(*Pi)y^/2. 
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Proof. The character uj determines an unramified character mapping through 
£(Fq). Specifically, a prime divisor with associated Galois orbit {Pi, . . . , Pd} 
contained in F(Fg) maps to the point P = Pi in F(Fq), and we define oj{^) — 
oj{P). The character thus defines a Galois character on the unramified cover 
defined by the isogeny £ ^ £ with kernel F(Fq). In particular the character is 
unramified and its conductor is trivial. Applying Proposition 1 we reduce to the 
consideration of the conductor of the character defined hy if o f. 

The character ifof defines a Galois character associated to an Artin-Schreier 
extension of F, as studied in Bombieri [2, Section VI] . In particular the conductor 
is determined in Theorem 5 of that work. The bound then follows from Propo- 
sition 2. □ 

In particular, from Theorem 1 and the identity (2) we see that the bound 

|-S’(w,'!/', /)! < 2deg(/)g^/^ (3) 

holds. If the polar divisor of / has support at a single prime divisor, then we 
have the stronger bound 

|5(o.,V',/)| <(l + deg(/))gi/2. 

For a subgroup of F(Fg) we define 

p&n 

f(P)¥^oo 



Corollary 1. Let f be a nonconstant function in Fg(£) and tp be a nontrivial 
character, then the bound 



|S'-h(w,' 0, /)! < 2deg(/)g^/^ 



holds. 

Proof. Let 17-^ C 17 be the set of characters x € 17 such that ker(x) contains H. 
Then is dual to £(¥g)/'H, so we may apply Lemma 1. Therefore 



Sn{t^,tp,f) 



^ E E xiPMPwm) 

Pe£{¥q)xeOn 

f(P)¥^'=o 



1 

\^n\ 



E 



^(x- /)■ 



Applying the inequality (3), we obtain the desired estimate. 



□ 
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4 Distributions of Points in Intervals 

We also require the following standard lemma, which appears, for instance, as 
Problem ll.c in Chapter 3 of [19]. 

Lemma 2. For any positive integers n, s, and r we have 

n—1 s+r 

EEe n{ak) \ < n{l + logn). 

k — 1 a—s 

We define an interval I in to be a subset of the form B + a[s, . . . , s + r] 
for an additive subgroup B of F^, an element a G F^, and nonnegative integers 
s and r. 

Lemma 3. For any interval I in F^ the hound 

EE V’(/3) < g(l + logp) 

/3g/ 

holds. 

Proof. For an additive subgroup B C F^, we define 'Fb = {if G F \ B C ker('0)}, 
and note that Fb is dual to ¥g/B. 

Now suppose / = B + a[r, . . . , r + s] , where i? C F^ is additive subgroup and 
a ^ B. Since ^ for all if not in Fb, we can express the sum as 

r+s r+s 

E E^(^) =E =1^1 E 

/3e/ i’&'I' /3GB k=r ip&'I'B k=r 

We set C = i? + oFp, and note that ififka) = 1 for all if in Fq. Therefore 

r+s 

E E^(^) =i^ii^'ci 

Tp&'I' ip&'I'B /'I'c k=r 

Since CfB = aFp is cyclic of order p and with dual group Fb/Fq, we can apply 
Lemma 2 together with = q/p to obtain the stated bound. □ 

For a character w G 17, a function / G Fg(F), and a subset S' C F^ we define 
the 

T(S, /, w) = {P G S(F,) I f{P) G S and ev{P) + 1}. 
and denote its cardinality by T(S, /, w). 

Theorem 2. Let E be an elliptic curve over a finite field F^, and let f he a 
function with poles only at O. Then for any interval I c¥q and character to of 
order m, the hound 

T{I, f, u;) - < 2(1 + deg(/))(l + logp)q^/^ 

m q 



holds. 




Exponential Sums 401 



Proof. Set Ti. to be the kernel of uj. Applying Lemma 1 we obtain the expression 

= E (EW(^^)-/^)) 

^ /3e/ Pe£{¥^) ipe'i' 

Pi^H 

= ^E E 

^ V'6!f' Pe£(F,) /36-f 

pi^n 

= - E (‘^(^o,'0, /) - Sn{ujo,%f,f)) ^V’(/3)"S 

^ Tp&'i' i3ei 

where wq G is the trivial character. Separating out the term corresponding to 
the trivial character fjo G 'P, we obtain the expression: 

^ ^ ^ t;:. fTi 

Applying Theorem 1 and Lemma 3 we obtain the desired result. □ 



Corollary 2. Let E he an elliptic curve over a finite field ¥ g of characteristic p, 
and take either f = x if p yf 2 or f = y if p 3 in Fg(f). Then for any interval 
I c¥q of cardinality greater than 5(1 + deg(/))(l + logp)q^^^, the set 

niJ) = {P€£{¥g)\f{P)€l} 



generates 

Proof. Since deg(x) = 2 and deg(y) = 3, we observe that the lower bound on / 
implies that |/| > |Fg| for q < 100. But for all q > 100, we note that the bound 

^ < g < 1 25 

N - g- 2(71/2 + 1 < ■ 

holds. Applying the bound of the previous theorem, we find that the subset 
T(/, /, w) of T(7, /), is nonempty for any nontrivial character uj. Therefore 
Til, f) is contained in no proper subgroup of F(Fg). □ 

5 The Algorithm 

Theorem 3. Given any £ > 0, there exists an algorithm which, given an elliptic 
curve £ over F^, constructs echelonized generators for £{¥q) in time 0{q^^'^'^^). 

Proof. For q large, the algorithm works by the following steps, and for small q 
we may solve the problem by any method we choose. 
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1. Find the group order N of i£(Fq), and factor it to find the set 
of all divisors. 

2. Construct the set T(/, /) of points P G F(Fg) with f{P) G /, 
for an appropriate choice of function / and interval I, such 
that T(7, /) contains generators for F(Fq). 

3. Reduce the generator set to a pair of echelon generators. 

The group order can be computed in polynomial time using the method of 
Schoof [11], with practical improvements by Atkin and Elkies [5]. The order can 
be factored by trial division in time but faster algorithms are also 

available [1,4], so this phase does not present the limiting complexity. 

By Corollary 2, if we set / equal to x for p ^ 2 or y if p = 2, then the set 
T(/, /) contains generators for F(Fg) for an interval I of size where 

0 < (5 < £. For each xq & I (or yo G I), the points (a;o, yo) in F(Fg), if such exist, 
can be found by solving a quadratic (or cubic) equation. Knowing a quadratic (or 
cubic) nonresidue, one can extract roots in polynomial time (see [1,4,16]). The 
nonresidue can be computed, for instance, by the 0((7^/^+‘^)-algorithm of [15], 
which finds a primitive root for F^. This one time computation has no impact 
on the complexity of the algorithm. Therefore the complexity of this stage of the 
algorithm is 

0(|T(/, f)\{logqf^^^) = 0{q^/^+^), 

which defines the complexity of the algorithm. 

Using the factorization of the order N, and a set of generators, we can find 
the exponent M of the group in polynomial time. If P is a point of order m and 
Q is a point of order n, where gcd(n, m) = 1, then P + Q has order nm. Thus it 
suffices to produce echelon generators for each subgroup Z/r'^Z x Z/r^Z, where 
r is prime and r'' and are the largest powers of r dividing M and L = N/M, 
respectively. Finding an element P of order involves only polynomial time 
group operations on elements of the set T (7, /) . Likewise a set of generators 
for the r^-torsion group can be produced in polynomial time, by multiplying 
points in T(7, /) by an appropriate factor. Setting Pi = r^^~^P, we take the 
Weil pairing of P\ with each element Q of order to identify an independent 
generator (see Menezes [10]). The complexity of this step is again 

o(|r(7,/)|(iog(?)°(i)) = o(gi/"+^) , 

so the complexity is as asserted. □ 

6 Remarks 

We note that the methods of this paper can be improved or extended in several 
ways. From the proof of Corollary 2, it is clear that the constant 5 in the bound 
can be improved to 4 + o(l). A more significantly improvement, however, is 
achieved using standard techniques (see Chalk [3]) to remove the logp from 
the bound. In another direction, combining the method of this paper with a 
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simple sieve method, it is possible to prove results on the distribution of points 
whose order equals the group exponent. In particular, for curves with cyclic 
point group i£(Fq), one obtains results on the distribution of cyclic generators in 
intervals. Since none of these results have consequence to the final complexity of 
the algorithm of this paper, we have left these results to comments. 

With minimal modification, the results of this paper carry over to a general 
result on Jacobians of a hyperelliptic curves over given by an equation of 
the form + a{x)y = b{x), where a(x) and b(x) are polynomials over F^. More 
precisely, it is possible to prove bounds on the size of sets of points on the curve 
which generate the group of rational points on the Jacobian. For elliptic curves, 
the Weil pairing is used to prove the independence of generators for the group 
of rational points [9,10]. Lacking an effective analogue of the Weil pairing, this 
approach seems to be the only available deterministic method for producing a 
provable set of elements generating the group. 

For finite fields of bounded characteristic there exist deterministic polynomial 
time algorithms for constructing a polynomial size set of elements containing 
a primitive element (see [12,13], and also Chapter 2 of [16]). It remains open 
whether similar improved bounds hold for the group of rational points on elliptic 
curves over finite fields of small characteristic. 

The bounds of exponential sums of this work also have implications for 
pseudo-random number generators. The bound of Corollary 1 has been used 
in [17] to show that the elliptic curve analogue of the Naor-Reingold pseudo- 
random function is uniformly distributed. Our results can also be used to prove 
that the elliptic curve analogues of the congruencial generator of pseudo-random 
numbers (see [6,7]) produce uniformly distributed sequences. 
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Abstract. Let / be a newform of weight 2 on To (IV), and let Af he the 
corresponding optimal Abelian variety quotient of Jo (A). We describe an 
algorithm to compute the order of the component group of A/ at primes p 
that exactly divide N. We give a table of orders of component groups for 
all / of level N < 127 and five examples in which the component group 
is very large, as predicted by the Birch and Swinnerton-Dyer conjecture. 



1 Introduction 

Let Xq{N) be the Riemann surface obtained by compactifying the quotient of the 
upper half-plane by the action of Ib(A). Then Xq{N) has a canonical structure 
of algebraic curve over Q; denote its Jacobian by Jo{N). It is equipped with 
an action of a commutative ring T = Z[. . ,T„ . . .] of Hecke operators. For more 
details on modular curves, Hecke operators, and modular forms see, e.g., [8]. 

Now suppose that / = On?” is ^ modular newform of weight 2 for the 

congruence subgroup ro{N). The Hecke operators also act on / by Tn{f) = a„/. 
The eigenvalues a„ generate an order Rf = Z[. . .a„ . . .] in a number field Kf. 
The kernel //of the map T —>■ Rf sending to a„ is a prime ideal. Following 
Shimura [15], we associate to / the quotient Af = Jo{N)/IfJo{N) of Jo{N). 
Then Af is an Abelian variety over Q of dimension [Kf : Q] , with bad reduction 
exactly at the primes dividing N. 

One-dimensional quotients of Jo (AT) have been intensely studied in recent 
years, both computationally and theoretically. The original conjectures of Birch 
and Swinnerton-Dyer [1,2], for elliptic curves over Q, were greatly influenced by 
computations. The scale of these computations was extended and systematized 
by Cremona in [6] . 

In another direction. Wiles [20] and Taylor-Wiles [18] proved a special case 
of the conjecture of Shimura-Taniyama, which asserts that every elliptic curve 
over Q is a quotient of some Jq{N); this allowed them to establish Fermat’s Last 
Theorem. The full Shimura-Taniyama conjecture was later proved by Breuil, 
Conrad, Diamond, and Taylor in [4]. This illustrates the central role played by 
quotients of Jo (AT). 
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2 Component Groups of Af 

The Neron model Aj'L of an Abelian variety ^/Q is by definition a smooth 
commutative group scheme over Z with generic fiber A such that for any smooth 
scheme S over Z, the restriction map 

Homz(S', A) HomQ(5'Q, A) 

is a bijection. For more details, including a proof of existence, see, e.g., [5]. 

Suppose that Af is & quotient of Jq{N) corresponding to a newform / on 
Ib(-/V), and let Af be the Neron model of Af. For any prime divisor p of N, 
the closed fiber Af is a group scheme over Fp, which need not be connected. 
Denote the connected component of the identity by A°f . There is an exact 
sequence ’’ 

with <pAf ,p a finite etale group scheme over Fp called the component group of A f 
at p. 

The category of finite etale group schemes over Fp is equivalent to the cate- 
gory of finite groups equipped with an action of Gal(Fp/Fp) (see, e.g., [19, §6.4]). 
The order of an etale group scheme G/Fp is defined to be the order of the group 
G(Fp). In this paper we describe an algorithm for computing the order of d>Af,p, 
when p exactly divides N. 



3 The Algorithm 

Let J = Jq{N), fix a newform / of weight-two for Ib(fV), and let Af be the 
corresponding quotient of J. Because J is the Jacobian of a curve, it is canon- 
ically isomorphic to its dual, so the projection J ^ Af induces a polarization 
A^f Af, where A'f denotes the Abelian variety dual of Af. We define the 
modular degree 5 ai of A f to be the positive square root of the degree of this 
polarization. This agrees with the usual notion of modular degree when 41/ is 
an elliptic curve. 

A torus T over a field fc is a group scheme whose base extension to the 
separable closure ks of fc is a finite product of copies of Gm- Every commutative 
algebraic group over fc admits a unique maximal subtorus, defined over fc, whose 
formation commutes with base extension (see IX §2.1 of [9]). The character group 
of a torus T is the group X = Homfc, (T, Gm) which is a free Abelian group of 
finite rank together with an action of Gal(fcs/fc) (see, e.g., [19, §7.3]). 

We apply this construction to our setting as follows. The closed fiber of the 
Neron model of J at p is a group scheme over Fp, whose maximal torus we 
denote by Typ. We define Xpp to be the character group of Typ. Then Xpp is a 
free Abelian group equipped with an action of both Gal(Fp/Fp) and the Hecke 
algebra T (see, e.g., [14]). Moreover, there exists a bilinear pairing 



{,) ■ ^J,p X ^J,p Z 
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called the monodromy pairing such that 

= coker(A’ j_p — > Hom(A’ j_p, Z)). 

Let Xj^p[If] be the intersection of all kernels ker(t) for t in If, and let 

Oif ■■ X,fp Hom(A’j,p[//], Z) 

be the map induced by the monodromy pairing. The following theorem of the 
second author [16], provides the basis for the computation of orders of component 
groups. 



Theorem 1. With the notation as above, we have the equality 

^ #coker(g/) ■ 5aj 

^ #{af{Xj,p)/af{Xj,p[If])) ■ 



3.1 Computing the Modular Degree SA,f 

Using modular symbols (see, e.g., [6]), we first compute the homology group 
iLi(Xo(fV), Q; cusps). Using lattice reduction, we then compute the Z-submodule 
Hi{Xo{N),Z] cusps) generated by all Manin symbols (c, d). Then Hi{Xo{N),Z) 
is the integer kernel of the boundary map. 

The Hecke ring T acts on Hi{Xq{N), Z) and also on Hom(iLi(Xo(fV), Z), Z), 
the linear dual, where t G T acts on ip G Hom(iLi(Xo(fV), Z), Z) by (t.(^)(a;) = 
ip(tx). We have a natural restriction map 

r/ : Hom(iLi(Xo(fV),Z),Z)[//] ^ Hom(iLi(Xo(fV),Z)[//],Z). 



Proposition 1. The cokernel ofvf is isomorphic to the kernel of the polariza- 
tion ^ Af induced by the map Jq{N) ^ Af. 

Thus the order of the cokernel of r/ is the square of the modular degree Sf. 
We pause to note that the degree of any polarization is a square; see, e.g., [13, 
Thm. 13.3]. 

Proof. Let S = S 2 {ro{N),C) be the complex vector space of weight-two modular 
forms of level N, and set H = Hi{Xq{N), Z). The integration pairing S x H ^ C 
induces a natural map 



: iL ^Hom(S'[//],C). 
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Using the classical Abel-Jacobi theorem, we deduce the following commutative 
diagram, which has exact columns, but whose rows are not exact. 

0 0 0 

1 t 1 

H[If] H ► •Pf(H) 

ill 

Hom(S', C) [//] ^ Hom(5, C) ^ Hom(S'[//] , C) 



A){C) ^ Jo(i)(C) ^/(C) 




By the snake lemma, the kernel of (C) — > Af{C) is isomorphic to the cokernel 
of the map H[If] — > <Pf{H). Since 

Aom{H / ker{<P f) , Z) = Hom(iJ, Z)[//], 

the Hom(— ,Z) dual of the map H[If] — > = H/keri^f) is rf, which 

proves the proposition. 

3.2 Computing the Character Group Xj^p 

Let N = Mp, where M and p are coprime. If M is small, then the algorithm of 
Mestre and Oesterle [12] can be used to compute Xj^p. This algorithm constructs 
the graph of isogenies between Fp-isomorphism classes of pairs consisting of a 
supersingular elliptic curve and a cyclic M-torsion subgroup. In particular, the 
method is elementary to apply when Xq{M) has genus 0. 

In general, the above category of “enhanced” supersingular elliptic curves can 
be replaced by one of left (or right) ideals of a quaternion order O of level M in 
the quaternion algebra over Q ramified at p. This gives an equivalent category, in 
which the computation of homomorphisms is efficient. The character group fbjp 
is known by Deligne-Rapoport [7] to be canonically isomorphic to the degree zero 
subgroup X{0) of the free Abelian “divisor group” on the isomorphism classes 
of enhanced supersingular elliptic curves and of quaternion ideals. Moreover, 
this isomorphism is compatible with the operation of Hecke operators, which are 
effectively computable in X{0) in terms of ideal homomorphisms. 

The inner product of two classes in this setting is defined to be the num- 
ber of isomorphisms between any two representatives. The linear extension to 
X{0) gives an inner product which agrees, under the isomorphism, with the 
monodromy pairing on fbjp. This gives, in particular, an isomorphism = 
coker(A’(0) — > Hom(A’(0), Z)), and an effective means of computing #coker(a/) 
and #(a/(Typ)/a/(A’yp[//])). 

The arithmetic of quaternions has been implemented in Magma [11] by the 
first author. Additional details and the application to Shimura curves, general- 
izing Xq{N), can be found in Kohel [10]. 
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3.3 The Galois Action on ^Af,p 

To determine the Galois action on $Af,p, we need only know the action of the 
Frobenius automorphism Frobp. However, Frobp acts on ^Af,p in the same way 
as —Wp, where Wp is the pth Atkin-Lehner involution, which can be computed 
using modular symbols. Since / is an eigenform, the involution Wp acts as either 
+1 or —1 on <l>Af,p- Moreover, the operator Wp is determined by an involution on 
the set of quaternion ideals, so it can be determined explicitly on the character 
group. 

4 Tables 

The main computational results of this work are presented below in two tables. 
The relevant algorithms have been implemented in Magma and will be made 
part of a future release. They can also be obtained from the second author. 



4.1 Component Groups at Low Level 

The first table gives the component groups of the quotients A/ of Jq{N) for 
N < 127. The column labeled d contains the dimensions of the Af, and the 
column labeled ^<pAf,p contains a list of the orders of the component groups 
of Af, one for each divisor p of N, ordered by increasing p. An entry of “?” 
indicates that p^ \ N, so our algorithm does not apply. A component group 
order is starred if the Gal(Fp/Fp)-action is nontrivial. More data along these 
lines can be obtained from the second author. 

4.2 Examples of Large Component Groups 

Let f^Af be the real period of Af, as defined by J. Tate in [17]. The second 
author computed the rational numbers L{Af,l)/flAf for every newform / of 
level N < 1500. The five largest prime divisors occur in the ratios given in the 
second table. The Birch and Swinnerton-Dyer conjecture predicts that the large 
prime divisor of the numerator of each special value must divide the order either 
of some component group d>Af,p or of the Shafarevich-Tate group of Af. In each 
instance d>Af ,2 is divisible by the large prime divisor, as predicted. 

5 Further Directions 

Further considerations are needed to compute the group structure of d>Af,p- How- 
ever, since the action of Frobenius is known, computing the group structure of 
d>Af,p suffices to determine its structure as a group scheme. 

Our methods say nothing about the component group at primes whose square 
divides the level. The free Abelian group on classes of nonmaximal orders of 
index p at a ramified prime gives a well-defined divisor group. Do the resulting 
Hecke modules determine the component groups for quotients of level p^M? 
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Component groups at low level 



N 


d 




N 


d 




N 


d 


^*^Af,p 


N 


d 


i^^Af,p 


N 


d 


H=d?Af,p 


11 


1 


5 




3 


13 


76 


1 


?,1* 


96 


1 


?,2 




3 


7 


14 


1 


6*, 3 


54 


1 


3*,? 


77 


1 


2*,1* 




1 


?,2* 


114 


1 


2*,5*,1 


15 


1 


4*, 4 
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3,? 
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3*, 2 


97 


3 


1* 




1 


20, 3*, 1’ 


17 


1 


4 


55 


1 


2,2* 
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6,3* 




4 


8 




1 


6,3,1 


19 


1 


3 




2 


14*, 2 




2 


2,2* 


98 


1 


2*,? 


115 


1 


5*,1 


20 
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?,2* 


56 


1 


?,1 


78 


1 


16*, 5*, 1 
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14,? 
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21 
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4,2* 
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?,1* 


79 
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1* 


99 
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?,1* 
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2 
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57 
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13 




1 


?,1 
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10,1* 
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?,1* 
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7,1* 
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7,1* 


58 


1 
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81 


2 


? 


100 


1 


? ? 

• 1 • 


117 
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27 
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? 
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10,1* 


82 
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2*,1* 


101 
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1* 
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7,3 


29 
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59 
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29 
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28,1* 
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25 
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30 
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61 
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83 
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1* 


102 


1 


2*,2*,1* 


118 


1 


2*,1* 
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5 




6 


41 
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119 


4 
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3*, 3 
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?,3 




2 
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104 
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?,1* 




5 


48*, 8 
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64 
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6,1* 




2 


?,2 


120 
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36 
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? ? 

• 1 • 


65 


1 
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86 


2 


21*, 3 


105 


1 


1,1,1 




1 


7,2,1 


37 
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1* 
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3*, 3 
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55,1* 
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10*, 2*, 2 
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4,1*,1* 


88 


1 


?,1* 




1 


24, 1* 




1 


? 


39 


1 


2*, 2 




1 


10,5,1 




2 


?,2* 




1 


3,1* 


122 


1 


4*,1* 




2 
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64,2* 
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?,1 
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1* 


51 
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3,1* 
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95,1* 
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2,1* 
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?,1* 
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21 
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16*, 4 


75 


1 


1*,? 
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94*, 1 


113 
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95 
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54*, 6 
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Large L{Af, 1)/J7aj 



N 


dim 




1154 = 2-577 


20 


2- 


1238 = 2-619 


19 


2- 


1322 = 2-661 


21 


2- 


1382 = 2-691 


20 


2- 


1478 = 2-739 


20 


2- 



L{Af,l)/nAf 

85495047371/17^ 

7553329019/5-31 

57851840099/331 

37-1864449649/173 

7-29-1183045463/5-37 



2- -17^-85495047371, 2' 
2--5-31-7553329019,2- 
2--331-57851840099,2- 
2- -37-173-1864449649, 2’ 

2- -5-7-29-37-1183045463, 2' 



Is it possible to define quantities as in Theorem 1 even when the weight of / is 

greater than 2? If so, how are the resulting quantities related to the Bloch-Kato 

Tamagawa numbers (see [3]) of the higher weight motive attached to /? 
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Abstract. Let x be a nontrivial Hecke character on a (strict) ray class 
group of a totally real number field L of discriminant du- Then, L(0, x) is 
an algebraic number of some cyclotomic number field. We develop an ef- 
ficient technique for computing the exact values at s = 0 of such Abelian 
Hecke L- functions over totally real number fields L. Let denote the 
norm of the finite part of the conductor of X- Then, roughly speaking, we 
can compute L(0, x) in elementary operations. We then 

explain how the computation of relative class numbers of CM-fields boils 
down to the computation of exact values at s = 0 of such Abelian Hecke 
L- functions over totally real number fields L. Finally, we give examples of 
relative class number computations for CM-fields of large degrees based 
on computations of L(0, x) over totally real number fields of degree 2 
and 6. This paper being an abridged version of [Lou4], the reader will 
find there all the details glossed over here. 

1991 Mathematics Subject Classification: Primary 11R29, 11R21, 
11Y35. 

Keywords and phrases: CM-field, relative class number, Hecke 
L- function. 



1 Notation 

Throughout this paper, we let L be a totally real number field of degree m > 1 
and J- be an integral ideal of L. We write for the set of all totally positive 
elements a of L such that iy-p(a — 1) > v-p{T) for all primes "P of L which divide 
T . The (strict) ray class group mod IF, which we denote by R;f(L), is defined 
to be the quotient of the group of fractional ideals of L generated by the primes 
not dividing IF, by the the subgroup consisting of all principal ideals (a) with 
a € Ljr. We let x denote a primitive character or order > 1 on R;r(L) and set 
fx = ^L/ q( 1F), the norm of the finite part of the conductor of x- We let M^/L 
denote the cyclic extension of degree and conductor T associated with x and 
we let denote the number of roots of unity of M^. We set = exp{2Tri/n^), 
Q(x) = Q(Cx)- We let (j)^ = (pin^) and Z[x] = denote the degree and the 
ring of algebraic integers of the cyclotomic field Q(x)> respectively. Finally, for 
any I relatively prime to we let ct/ denote the Q-automorphism of Q(x) which 
is defined by 

W. Bosnia (Ed.): ANTS-IV, LNCS 1838, pp. 413-422, 2000. 

© Springer- Verlag Berlin Heidelberg 2000 
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2 Computation of X(0, x) 

The first aim of this paper is to develop a practical and efficient technique for 
computing the exact values at s = 0 of such Abelian Hecke L- functions, i.e. for 
computing the exact values of the rational coordinates of this algebraic number 
L(0, x) in a given basis of this cyclotomic field Q(x) (see Theorem 5 and Remark 
1). To compute such exact values we fix a Z-basis B of the ring of algebraic 
integers Z[x] of the cyclotomic field Q(x) generated by the values of x and we 
compute the coordinates of L{0, x) in this basis B. Since these coordinates are 
rational numbers whose denominators are bounded beforehand (see Theorem 
1), to compute their exact values we only have to compute sufficiently good 
numerical approximations of them. By expressing these coordinates as linear 
combinations of finitely many values L(0,xO for some I > 1 (see (6) and (7)), 
we will reduce the computation of approximations of these coordinates to the 
computation of sufficiently good approximations of values of several L{0, xO- 
For each X G R-f(L), the partial zeta function of X is defined for 3?(s) > 1 by 
C:f(X, s) = where the summation is taken over all integral ideals 

A of L, prime to which belong to the class of X, and where N(A) denotes 
the norm from L to Q of A. Siegel and Klingen who proved that 0) is 

rational. Let now x be a primitive character of order > 1 on the (strict) ray 
class group R;f(L) modulo T and let 

L{s,x) = ( 1 ) 

I 

(where X ranges over a set of representatives of the ray class group modulo T) 
be the Abelian Hecke L-series associated to x- Setting 

a«(x) = XI (2) 

N(A)=n 

(this sum ranges over all the non zero integral ideals of L of norm n) we have 

i(AX) = X^ (5ft(s)>l) (3) 

n>l 

According to (1) and to Siegel-Klingen’s Theorem, L(0,x) is in Q(x) and for 
any rational integer I relatively prime to we have 

CT/(X(0,x)) = ^(0,xO- (4) 

Theorem 1. (See [CS] and [Cas]). It holds w^L{Q,x) ^ Z[x]. 

Let B = {ei, • • • , be any Z-basis of Z[x]- Let B^ = {6*i, • • • , 9^^} be its 

dual basis relative to the trace form (see [Lan, Prop. 2 page 58]), hence 



J 1 iik = l 
|o iik^l 
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and set 

M{B^) = max Wi{0j)\. (5) 

l<t<n^ , gcd(i,n^) = l 

Theorem 2. Let T he a non zero integral ideal of a number field L of degree m > 
1. Let X primitive character on the (strict) ray class group R;f(L) modulo 
T and let f^ = -/Vl/q(J^) denote the norm of the finite part of the conductor of 
X- Let B = {ei, • • • , e<^^} he a Z-basis of Z[x] and let B^ = {6*i, • • • , 04 ,^} he its 
dual basis relative to the trace form. Define rational integers b^{k) by 



WxH^^x) = '^bx(k)ek G Z[x] 

fc=i 

(see Theorem 1). We have 

b^{k) = w^ ^ cr/(6lfc)T(0,x') 



1 = 1 

gcd(Z,nj^-) = l 



and these coordinates b^{k) are rational integers of reasonable size: 
\b^(k)\ < 2w^M{B^)y^[^log{dU^)) . 

2.1 Numerical Computation of Approximations of T(0, x) 
Theorem 3. For m > 1, j G {1, 2}, i? > 0 and a > 1 we set 

1 pa+iao TD2 — 2S 



(which is real and does not depend on a > \). Then 
0 < Km,2{B) < Kra,i{B) < me 



-B^l" 



( 6 ) 



( 7 ) 



(8) 



(9) 



Let X be a primitive Abelian Hecke character over a totally real field L of degree 
m. Set = yJd-Lf^l'K'^ . Assume that x is ramified at all the m infinite places 
ofh, let the Unix) 's be as in (2) and set 



5 ^ m ( x ) - ^ Km, 2 {^/A^) -\- 2 2^ ^ Km,i{n/A^). 



M 



^ ji 

n—1 



n—1 



For any positive integer M > we have: 



2m 



\m x) - Sm{x)\ < ^GlAlog(Me) + mV 2 )™e-(“/^A 



2/n 



(10) 
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It now remains to explain how we compute numerically Km,i{B) and Km, 2 (B) 
for i? > 0. We give a precise result for the case m = 2: 



Theorem 4. Let 7 = 0.577 215 664 901 532 • • • denote Euler’s eonstant and set 
Ai = 1, A 2 = ttB. For B > 0, we have: 



K2Ab) = a,+a E 7 + log i? - 



n>0 



2n 



+ 3 - j ^ fcy (2n + 3 - j){n\y 



and for any integer M > 0 we have \Rj{M)\ < 2i?^^+^/(M + 1)(M!)^ where 



R,{M)= Y. (7 + logB- 



-Ei 



j^2n+2 



n>M 



2n + 3 — j (2n + 3 — j)(n!)^ 



Of course, one must finally know how to compute the coefficients a„(x). Since 
n Unix) is multiplicative, one needs only explain how to compute Upk (x) (and 
we refer the reader to Proposition 1 for such an example). 



2.2 Numerical Computation of the Exact Values of the 

Theorem 5. Let A > 1, n > 1 and a Z-basis B of the ring of algebraic integers 
of the cyclotomic field Q(Cn) be given. Let M{B^) be as in (5). Let x range over 
the primitive characters of order n^ = n on (strict) ray class groups and let the 
b^{k)’s be as in (6). Using (10) and (1) we obtain 

Ki.k) = w^ ^ ai{9k)SM{x') ( 11 ) 

1=1 ^ ^ 
gcd(Z,nj^-) = l 

Therefore, for A^ large enough, the coordinates b^{k) in the basis B of the alge- 
braic integer w^L{Q,x) G Z[y] are rational integers which can be determined in 
0{A^^~^’’) elementary operations by computing the 4>{n) approximations Sm{x’‘) 
for M equal to the least integer greater than or equal toA^(Xlog and for 

I in the range 1 <l <n and gcd(^, n) = 1. 



Remark 1. Here we assume that is known beforehand, for its computation 
from its definition requires more than elementary operations. However, for 
certain classes of characters is indeed known beforehand (see [FQ], [Fro], 
[Lou3], (16) and (18) in Section 4). In the case where it is not known be- 
forehand we explained in [Lou2, Section 5]) how to compute at the same time 
numerical approximations of and T(0, x) to end up with a practical tech- 
nique for computing the exact value of L(0, x) which conjecturally requires only 
elementary operations. 
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3 Relative Class Numbers and X-Functions at s = 0 

Let N be a CM-field. Then N is a totally imaginary number field which is a 
quadratic extension of its maximal totally real subfield N+. Let n denote the 
degree of N+. Let h^, Qn G {1,2} and wn denote the relative class number of 
N, the Hasse unit index of N and the number of roots of unity in N, respectively. 
Let L be any subfield of N+ such that the extension N /L is Abelian, and let m 
denote the degree of L. We thus have the following lattice of subfields: 

m n/m 2 

Q L — N+ N 

We can always choose L = N+. However, the smaller is the degree m of L 
the more efficient is our technique for computing K^. Therefore, we will choose 
L = Q whenever N is Abelian, whereas we will choose for L the only real 
quadratic subfield of N over which N is cyclic whenever N is a dihedral CM-field. 
We let An /l denote the group of primitive Abelian Hecke characters associated 
with the Abelian extension N /L and set Ap^^^ = /l \ A"n+ /l • We have: 

n 2-™L(0 ,x) (12) 

Now, let us say that x' G An/l is equivalent to x G An/l if there exists I 
relatively prime to the order of x such that x' = X^- Notice that if x! is 
equivalent to x then x! and x both have the same order and conductor. We let 
^N/L denote any set of representatives of the set of equivalence classes of Ap^^j^ 
modulo this relation. According to (4) and (12) and noticing that for x G An/l 
we have L C C N (which implies \ wn), we have: 

Theorem 6. For any x G ^n/l have wnL(0,x) G Z[x] and 

^N = QnWn Aq(^)/q^2“™L(0, x)) , (13) 

and by computing with large rational integers we can easily compute the (usually 
very large) exact values of the rational numbers Aq(^)/q (2“™L(0, x)) as soon 
as we have computed the (small) coordinates b^{k) G Z of w^L{Q,x) 

Remark 2. Our present method for computing relative class numbers is much 
more efficient than the method developed in [Lou2, Theorem 7]. There, we had 
to compute very good approximations of all the 5 m (xO (defined in Theorem 3) 
prior to taking their product to deduce the value of a relative class number. Here, 
we only have to compute fair approximations of these 5 m (xO prior to taking 
linear combinations of them to deduce the exact values of the coordinates of 
L(0, x)- Then, we compute the value of the relative class number by computing 
the norm of the algebraic number L(0, x)- 
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4 Examples 

4.1 Relative Class Numbers of Some Dihedral CM-Fields 

Let p > 3 be an odd prime. Let N be a normal CM-field of degree Ap whose Galois 
group is isomorphic to the dihedral group D^p of order Ap. Hence, N = N+M 
where M an imaginary biquadratic bicyclic field and N+ is a real dihedral field 
of degree 2p, cyclic of degree p over the real quadratic subfield L of M. There 
exists a positive rational integer / > 1 such that the conductor 
extension N+/L is equal to the ideal (/) of L (see [Mar] and [LPL]). We proved 
in [LOO] that Qn = Qm, wn = wm and that divides h^. Hence, formula 
(13) applied to both N and M yields 

^n/^m = -^Q(Cp)/q(4'^(®’ 

Here, y is any one of the p — 1 primitive characters of order 2p associated with 
the cyclic extension N/L. Hence, = wn = wm divides 12. Choose B = 

{Cp, • • • , Then = {(Cp ^ - l)/p, • • • , (Cp - l)/p} and 

p-i 

icmL(0, x) = ^ 6;,(fc)Cp' G Z[x] = Z[Cp] (14) 

k^l 

with6;,(fc) = ^ E (Cp-"^"'+'^-l)L(0,x"'+') (15) 

^ 1=0 

l^{p-l)/2 



(use (7)). Since the induced characters (x^^'*’^)* of the dihedral group Gal(N/Q) 
of order 4p are real valued, we have b^{p — k) = b^{k) for 1 < fc < (p — l)/2, 
L(0, = L(0, x^^'*’^) are real, and thanks to the computation of good 

enough numerical approximations of the L(0,x^^~'’^) for 0 < ^ < (p — 3)/2 we 
can use (15) to compute the exact values of the coordinates b^{k) of L(0,x) G 

Q(Cp)^ = Q(cos( 27 t/p)) and h^/h^ '^= (^n/m)^ is a perfect square with 

^N/M “ ■^Q(Cp) + /q(4'^(®’ 

Moreover, we have (see [FQ]): 

= +1. (16) 

To make our construction of x easy, we will choose an example such that 

(i) L has class number one, 

(ii) M/L is unramified at all the finite places, 

(iii) the conductor (/+) of the cyclic extension N+/L of degree p is of the 
form (q) for some prime rational number q. 

In that situation, x is primitive Abelian Hecke character of order 2p on the 
ray class group of conductor T = (q) of L and there exists a character x+ on 
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(Al/(( 7 ))* of order p (and trivial on the image of Z in this group) such that for 
any a yf 0 in the ring of algebraic integers Al of L we have x((a)) = v{a)x+{o:), 
where is the sign of the norm of a. 

Example. Choose p = 41, L = Q(-\/^), let N+ be the only real dihedral field of 
degree 2p = 82 for which i^N+/L = (?) = (2297) and take M = Q(-y/^, 23), 

for which = 1. Hence, N = N+M is a dihedral CM-field of degree Ap = 164. 
Since {q) = QQ' splits in L then (see [Lou2] and [LPL]): x((a)) = v{a)(j){a/ a') 
for some character (j) of order p = 41 on the cyclic group (Al/Q)* of order q — 1, 
group which is canonically isomorphic to {Ti/qTi)* (here a' is the conjugate of a. 
in L). To make it explicit which x we used we chose Q = ( 7 Z + ((2527+-\/3l)/ 2)Z 
and (j) is the one for which (j){5) = Qp = exp(27ri/41). According to our numerical 
computation we have 

40 

i(o,x) = E^fc^« 



with = bk and the following Table: 



k 


1 2 3 4 5 


bk 


-4008 -4000 -4028 -4076 -4260 


k 


6 7 8 9 10 


bk 


-4092 -4100 -3964 -3664 -3868 


k 


11 12 13 14 15 


bk 


-3820 -3964 -4024 -4044 -4700 


k 


16 17 18 19 20 


bk 


-4384 -4012 -4068 -3960 -3896 



Hence, h^/h^ = (/in/m)^ = 47806 51139 18289 69370 25122 72645 

03025 58591 42700 36539 28149 96559 « 4 • lO^^. 



4.2 Relative Class Numbers of Some CM-Fields of Degree 24 



Let N be a normal CM-field of degree 24 with Galois group isomorphic to 
SL 2 {F 3 ), the special linear group over the finite field with three elements 
(see [Loul, Section 5] and [LLO]). Then N+/Q is a normal extension of degree 
12 with Galois group isomorphic to the alternating group A 4 of degree 4 and 
order 12, and N is a quaternion octic extension of some cyclic cubic field F. Let 
/f denote the conductor of F. We let L/F denote a fixed quadratic subextension 
of the three quadratic subextensions L^/F of the bicyclic biquadratic extension 
N+/F. (Notice that L/Q is not normal and that the three are conjugate). 
Then, N/L is a cyclic quartic extension and we let x denote any one of the two 
conjugate characters of order four associated with this cyclic quartic extension 
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N/L. An (incomplete) lattice of subfields is given in the following Diagram: 

Gal(N/Q) = SL 2 {F 3 ) 

Gal(N/F) = Qs 
Gal(N+/Q) = A 4 
Gal(F/Q) = C3 
Gal(N/Li) = C4 

Q 

and we have 

= QNWN-/Vq(i)/Q x)) ■ 

Now, since F is the maximal Abelian subfield of N then wn = 2 and 2L(0, x) G 
Z[z]. Since the character of Gal(N/F) induced by x is the irreducible character 
of degree two of the quaternion octic group which is real valued, then L(0, x) is 
real, hence L(0, x) G 5Z and 

/iN = (gN/2)(L(0,x)/32)2, (17) 

which implies L(0,x) G 32Z, and A(0,x) = x)/’’’^ < 0- Moreover, if 

is odd then Qn = 2 , = (A(0,x)/32)^ is a perfect square and L(0,x) ^ 

64Z. Gonversely, if L(0, x) G 32Z \ 64Z then Qn = 2, is odd and = 
(A(0, x)/32)^ is a perfect square. Now, for any number field E, let CIe and Cl^ 
denote the 2-Sylow subgroups of the ideal class group and narrow ideal class 
groups of E, respectively. According to [LLO], if is odd then CIe and Clp 
are both isomorphic to ^ CIe is isomorphic to Z/2Z, Cl^ is isomorphic 

to Z/4Z and N+ is the Hilbert 2-class field of F and N is the Hilbert 2-class 
field of L. In particular, = /p/zr^ and 

= -1 (18) 

(for the Abelian extension N /L is unramified at all the finite places of L, but 
ramified at the six infinite real places of L). 

Now, using class field theory, the reader would prove the following result 
which, together with the results of section 2.1, enables us to compute the exact 
values of L(0, x) for such characters x^ 

Proposition 1. Let F, L, N and x o,s above. Assume that /p = 9 = 1 
(mod 6) is prime, that the narrow and ordinary class groups o/F are isomorphic 
to {T^jTL')^, and that the narrow and ordinary class groups o/L are isomorphic 
to Tjj/CL and T^jTL, respectively (see [Loul, Pro. 16]). 





Fast Computation of Relative Class Numbers of CM-Fields 



421 



1. if (p) = P is inert in F then (p) = PAl = VV' splits in L/F, and setting 
Cp = xi'P) = xi'P') = ± 1 ; have 

fO if 3 does not divide k 

^ \ Cp^^(fc + 3)/3 if 3 divides k 

and Cp = +1 if and only ifV is principal in the narrow sense (notice that V 
is always principal in the ordinary sense). 

2. if p = q is totally ramified in F, say pAp = P^, then PAp = VV' splits in 
L, and setting Cp = x{V) = x{V) = ±1, we have 

apt^ix) = ep(fc+ 1) 

and Cp = +1 if and only ifV is principal in the narrow sense (notice that V 
is always principal in the ordinary sense). 

3. Assume that {p) = P 1 P 2 P 3 splits in F. Then either these three ideals are 
principal in F or none of them is principal in F. 

(a) If the three Pi’s are principal in F then each PiAp = V{P'i splits in 
L/F, and setting Cp = x{Vi) = x{V'i) = ±1 which does not depends on 
i, we have 

ttpk (xO = ep(fc + 5)(fc + 4)(fc + 3)(fc + 2)(fc + 1)/ 120 

and Cp = +1 if and only if Vi is principal in the narrow sense (notice 
that Vi is always principal in the ordinary sense). 

(b) If none of the Pi is principal in F then two of these prime ideals PiAp = 
Vi and P 2 AL = V 2 are inert in L/F and the third on e PaA p = P 3 P 3 
splits in L/F. We have x{Vi) = x{V 2 ) = —1, xiV'^) = xiVs) = and 

, , _ J 0 if k is odd 

V U) = I (_i)fc/ 2 ((fc/ 2 ) + i)((fc/ 2 ) + 2)/2 if k is even. 

For example, using Pari to decide whether a given ideal V of the sextic field 
L is principal in the ordinary or narrow senses, we computed the following Table 
of relative class numbers: 



/f 


Pf{X), Pl(X) 


o' 




163 


Pf{X) = x'^' — x‘‘ — 54a; + 169 








Pf(X) = - 3a:® - lla;^ + 27a;® - 3a;^ - 11a; + 1 


-32 


1 


349 


Py{X) = a;® - a;^ - 116a; + 517 








Pl(N) = a;® - 3a;® - 17a;^ + 39a;® - 3a;^ - 17a; + 1 


-96 


32 


397 


Py(X) = a;® - a;^ - 132a; + 544 








Py{X) = a;® - 26a;^ + 93a;^ - 4 


-96 


3^ 


853 


Py\x) = a;® - a;^ - 284a; - 1011 








Py{X) = a;® - 3a;® - 53x^ + 111a;® + 705a;^ - 761a; - 91 


-352 


11^ 


937 


Py\x) = a;® - a;^ - 312x + 2221 








Py Ix) = a;® - 3x® - 29a;^ + 63a;® - 3a;^ - 29a; + 1 


-608 


19^ 



Notice that in these five cases it holds L(0, x) G 32Z \ 64Z. Hence, Qn = 2 (by 
(17)) and hf^^ — (L(0, y)/32)^ is a perfect odd square. Such computations for the 
23 CM-fields N associated with the 23 cyclic cubic fields F whose conductors /f 
are listed in [Loul, Prop. 16] enable us to prove: 
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Theorem 7. There exists only one normal CM-field of degree 24 with Galois 
group isomorphic to SL 2 {F^) with class number one: the CM-field associated 
with the cyclic cubic field F of conductor /f = 163. 

4.3 Relative Class Numbers of Some CM-Fields of Degree 42 

We refer the reader to [LPCK] for examples of computation of Artin root num- 
bers and values at s = 0 of L-functions associated with characters of order 
14 on ray class groups of real cyclic cubic fields L. These computations are used 
to prove the following result similar to Theorem 7 : 

Theorem 8. There is no non- Abelian normal CM-field of degree 42 with relative 
class number one. 
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Abstract. We will investigate two well-known square root finding al- 
gorithms which return the roots of some quadratic residue modulo a 
prime p. Instead of running the mechanisms modulo p we will investi- 
gate their behaviour when applied modulo any integer n. In most cases 
the results will not be the square roots, when n is composite. Since 
the results obtained can easily be verified for correctness we obtain a 
very rapid probable prime test. Based on the square root Ending mech- 
anisms we will introduce two pseudoprimality tests which will be shown 
to be extremely fast and very efficient. Moreover, the proposed test for 
n = 1 mod 4 will be proven to be even more efficient than Grantham’s 
suggestion in [5]. 



1 Background and Motivation 

Two classical problems in number theory at first seem to be unrelated: the 
computation of square roots modulo a prime p, and the determination of whether 
or not a given number n is a prime. 

A great number of suggestions have been made for efficiently solving these 
two problems. The operation of computing square roots modulo p can be 
performed with expected running time 0((lgp)^) (cf. [8]), respectively 0((lgp)^) 
(cf. [9]) bit operations. On the other hand, the primality testing problem for 
arbitrary large numbers n is still considered to be a difficult one. Establishing a 
deterministic answer concerning the primality of any large n seems to be very 
expensive. Indeed, in practice, probable prime tests are frequently used, which 
‘only’ yield a correct answer with some specific probability that depends on the 
primality testing condition used. 

Most of the pseudoprimality algorithms originate in some sense on Fermat’s 
Little Theorem = 1 modp, for any base a G Z*, respectively their speci- 
fications based on Euler’s criterion, modp, or on the stronger form 

= 1, respectively = —1 mod p for some 0 < j < r — 1 where p — 1 = 2’' s 
with s odd. Although exponentiation modulo p can be performed extremely fast, 
the catch of the test is that they allow pseudoprimes, i.e. composite numbers that 
pass the corresponding conditions. 

* Research supported by the Austrian Science Fund (FWF), FWF-Project no. P 

13088-MAT 



W. Bosnia (Ed.): ANTS-IV, LNCS 1838, pp. 423-437, 2000. 
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In order to minimize the probability of encountering pseudoprimes, sugges- 
tions have been made to replace the testing condition based on the power poly- 
nomials with other functions that have suitable properties. These include the 
use of the roots a,a G Fp 2 of some polynomial f{x) = — Px + Q. It turns 

out, that to some extend this approach can be viewed as an analogue of the 
Fermat based tests in terms of arithmetic in the quadratic extension 
field Fp 2 . Clearly, when p is a prime, = 1 in Fp 2 where here and in the 

following a. = a{P, Q) denotes any of the two roots in Fp 2 . However, it turns out 
that the exponent — 1 is too large to establish very strong primality criteria. 

In particular, a crucial refinement can be made by observing that a condition 
involving lower exponents can be utilized. In fact, it is well known that = 

1 , respectively Q mod p, according as D = P^ — 4Q is a residue, or nonresidue 
modulo p. Obviously, the former case reverts us back to the original Fermat 
condition, so any improvement has to be based on the latter case. The quantity 
Q, respectively 1, is often called a general multiplier (cf. [15]) of a modulo p. 
Indeed, this additional specific value Q, when does seem to play a 

crucial role in quadratic field based primality testing. 

For simplicity we sometimes write e(p) = e(n) = (-^), when n 

is any odd integer. Clearly, the condition = Q, respectively 1 mod n 

can now be used as a primality testing condition. However, also pseudoprimes 
based on this approach are known (cf. [3,5]), also when e(n) = —1. They are 
often referred to as QF-based (quadratic field based) pseudoprimes, in short 
QFpsp{Q), or Frobenius pseudoprimes, with respect to the general multiplier 
Q. 

Although the above two major methods of probable prime testing yield a 
number of pseudoprimes, the combination of these two types of tests is very 
effective (cf. [2,5]). No composite number is known yet that passes both a Fer- 
mat based test and a QF-based test for prescribed parameter searching routines 
for P,Q (cf. [2]) such that e(n) = —1. Grantham [5] also established strong 
theoretical results demonstrating that such types of pseudoprimes are very rare. 

The combination of the Fermat- and the QF-based tests of course can even 
be made more powerful when on both sides the stronger versions are being used. 
This is actually the basis of Grantham’s extremely efficient probable prime test. 
As in the Miller-Rabin test, Grantham’s basic idea relies on the fact that the 
square roots of 1 in the field Fp 2 can only be 1 or —1. Alternatively, we can 
interpret this as follows. For the roots a G Fp 2 with e(p) = —1 the quantities 
a^, respectively mod p for some 0 < j < r need to be elements of Z* where 
now p -h 1 = 2’~s with s odd. Furthermore, if = 1, the latter condition can 
be sharpened to j < r — 1 . 

However, it is not known what these values in the prime field Fp actually 
need to be. Not even do we know these values in terms of the generalisation 

p ^ 1 

of Euler’s criterion in the extension field. By the Euler condition, a~^ 
has to be equal to the Legendre symbol . On the other hand, for the QF- 
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based tests, although is known to be equal to the general multiplier 1, 

respectively Q, the immediate question arises, what the explicit value of a’’ s'" 
is when reduced modulo a prime p. In other words, we are interested in the 
generalisation of the Legendre symbol ■ 

This brings us now back to the problem of the computation of square roots. 
Actually, it is known (cf. [9]) that if Q is a quadratic residue and e{p) = — 1 
that the quantity is a square root of Q modulo p. But we don’t know 

which one. Although in the context of finding square roots this does not matter, 
when applied to pseudoprimality testing, a specific answer would enable us to 
establish more stringent testing conditions. 

Outline of the Paper: In the first part of this contribution we actually exhibit 
the correct sign and a specific formula for mod p which will indeed turn 

out to be the exact generalisation of Euler’s criterion. That is, for e{p) = — 1 we 
will explicitly evaluate mod p via the ‘ordinary’ Legendre symbol modulo 

p with respect to the parameters P, Q. 

Consequently, we then utilize properties of square roots modulo primes p as a 
primality testing condition. We will show that the square root algorithms when 
applied to any composites n rather than primes will usually not return the correct 
root modulo n. Furthermore, we introduce two pseudoprimality tests based on 
the square root finding algorithms that will be shown to be very efficient. 

Indeed, the proposed test for n = 1 mod 4 will be proven to be stronger 
than Grantham’s [5], which presently is the most efficient probable prime test 
known. Our test has comparable running time, but we will be able to establish a 
tighter bound on the probability that a composite number fails to be identified 
as such by the test. Moreover, we believe that our test is easier to describe than 
Grantham’s. 

2 Square Roots by Utilizing Quadratic Fields 

It is known that the Legendre symbol can be evaluated very efficiently for 
any Q coprime to an odd prime p. Consequently, deciding whether or not Q is ^ 
square modulo p can be worked out very fast. For actually exhibiting a square 
root of a quadratic residue Q in Fp a number of algorithms have been proposed. 
In detail, if p = 3 mod 4, this can be achieved very rapidly by means of the square 
& multiply mechanism. By Euler’s criterion we have mod p, and 

thus, if = 1, one readily verifies that 

p-Fl 

(±Q^)modp (1) 

are the two square roots of Q modulo p. 

When p = 1 mod 4, this method obviously cannot be applied. However, when 
working in the quadratic extension Fp 2 of Fp analogous results may be obtained 
(cf. [9]). 
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2.1 Some Fundamental Properties 

We will now investigate properties of the elements in the extension field, as this 
will turn out to yield our fundamental results. 

We consider the roots a, a € Fp 2 of f{x) = — Px + Q modulo p. Let 

D = P^—4Q be the discriminant of f{x) and let e{p) = denote the Legendre 
symbol. We will assume throughout that e{p) yf 0. As noted above, = Q, 

respectively 1 modp, according as e{p) = —1 or 1. Moreover, if = 1 then 

P-e(p) P-e(P) , 1 *1 T / 0\ -I P-^(P) P-e(p) , 2 f, 

a 2 = Q. 2 mod/?, wniie ii ( I = —1, a 2 = —a 2 mod/? (ci. 

[18]). Thus, in order to establish an analogue of Euler, which holds for any of 
the roots a, a, it will only make sense to consider the case that = 1. 

Our first step is to establish some link between the roots in Fp 2 and the 
elements P and Q in the prime field. 

Lemma 1. Let a G Fp 2 be any root of x"^ — Px + Q and suppose that a is a 
square root of Q modulo p. Then {P + 2a)a = (a + a)^ mod p. 

Proof. By Vieta’s rule we have + Q = a{a + a) and therefore, {a + a)^ = 
aP + 2aa mod p, as claimed. □ 

When working in the extension field, we thus have found a square root of (P+ 
2a)a. However, our basic interest is the calculation of square roots of quadratic 
residues Q in the prime field Fp. That is, we only utilize the arithmetic of the 
extension to obtain our desired results modulo p. We still need the following. 

Lemma 2. If a, a are the roots of x^ — Px + Q modulo p and a is a square root 
of Q modulo p then a + a and a + a are the roots of x^ — {P + 2a)x + {P + 2a)a = 
0 mod p, respectively. 

Proof. This follows immediately, since the discriminants P^—AQ and (P+2a)^ — 
4(P+2a)a of the two characteristic equations under inspection are the same. □ 

2.2 The Generalisation of Euler’s Criterion 

Theorem 1. If a is any root of x"^ — Px + Q, where of = Q mod p then 

modp, z/e(p) = l, 

amodp, if e{p) = —\. 

Proof. Consider the root a' = a + a mod p which is by Lemma 2 a root of 
x'^ — P'x + Q', where P' = {P + 2a), Q' = {P + 2a)a mod p. Then, by the 
Frobenius automorphism we have ( 0 + 0 )^“*^^^^ = 1, respectively (P+2a)a mod p, 

according as e(p) = ^ = 1 or —1. We now conclude that 

{a + = ((P + 2a)a) ^ mod p. 
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By means of Lemma 1 we also have 



((^ 






P-e(p) 



= {P- 



s p-e(p) p-e(p) . 

2a) 2 q; 2 mod p, 



and therefore, when combining these results, 



, -r^ „ - l-g(P) „ ' P-g(p) P-g(P) , 

(P+2a) 2 a 2 =(P+2a) 2 q, 2 modp. 



which gives the formula above. The result does not depend on which of the roots 
±a of Q modulo p is being selected. Clearly, the above also holds when a is 
replaced by —a mod p in the formula above. □ 

|^ (P+2a)a-(P-2a)(-a) ^ ^ = (^) following. 

Corollary 1. Let e{p) = — 1. Then a” 2 '”' = modp, when 

p= 3 mod 4, and a” 2 *”’ = ^ ^ ) = — ( ^ P> when p = l mod 4. 

In particular, for any P with Theorem 1 now yields 

the QF-based method for calculating the square root of Q modulo p. 

Corollary 2. Let = b mod {p, x'^ — Px + Q) where = ^(p) = 

— 1 modp. Then b is a square root of Q modulo p. 



3 Applications to Pseudoprimality Testing 

3.1 Pseudoprimes to the Basic Square Root Tests 

We now apply the above results as a probable prime testing function. If n is 
indeed an odd prime, then the properties established above need to be fulfilled. 
On the other hand, n might fulfill these attributes, although being composite. In 
other words, there might be some composite numbers that do return the square 
root of Q modulo this integer n. That is, the above algorithms, when applied 
modulo n, return the same result as if the square root of Q were calculated 
modulo each prime factor of n and then combined by means of the Chinese 
Remainder Theorem. 

We will now investigate these pseudoprimes with respect to the above square 
root algorithms (1) and Corollary 2. Most interestingly, they will correspond to 
those based on the Fermat- and on the QF-based (or, analogously, Lucas based) 
probable prime tests, respectively (cf. [14]). 

For the different notions and the correspondence of the types of pseudoprimes 
encountered, we will refer to [14] and [6]. As with all probable prime tests, our 
goal is to establish some testing conditions that yield strong pseudoprimes. That 
is, the composite numbers that pass the testing condition should satisfy a number 
of strong and well specified properties. 
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Lemma 3. Let Q G with = 1 where n = 3 mod 4 is a composite 

integer. Then a = mod n fulfills of = Q mod n iff n is an Epsp{Q), i.e. 
when Q^~ = 1 mod n. Moreover, if a is the correct root of Q modulo n, then n 
is also a psp{a), i.e. = 1 mod n. 

Proof. If the algorithm does calculate the correct root modulo n, then we have 
Q^~ = of mod n, or alternatively Q^~ = 1 = mod n. The conversion of 
the statement, as well as the second assertion, is immediately obvious. □ 

The underlying fact of our investigations in the quadratic field relies on the 
property that although when the roots a,aofx‘^ — Px + Q are elements of the 
quadratic extension (i.e., when e(p) = —1), certain powers thereof yield elements 
in the ground field Fp. 

Generally, for q = p^ , any integer R such that = S for some nonzero 

element S in F^, is called a general restricted period with respect to P and 
Q. The element S is called a general multiplier corresponding to R (cf. [15]). 

These two quantities have been specified above modulo primes p. When work- 
ing modulo composite numbers n we will show below that the corresponding 
criteria actually provide good primality testing conditions. 

Lemma 4. Suppose n is a composite integer, = 1, and (-^) = —1 mod n. 

If b = x^ mod (n, x^ — Px + Q) is such that = Q mod n and ( ) = 1, 
then R = is a general restricted period with respect to P and Q modulo n 
and b = ( ^ ) a is the general multiplier corresponding to R. Consequently, n 

is a QFpsp{Q), i.e., = Q mod n. 

Proof. By means of the arithmetic modulo any prime factor p of n, and modulo 
f{x) = x"^ — Px + Q it follows that = b (mod p) for both roots 

a and a of f{x). Suppose the test passes for the wrong sign of b. That is, 

= 1 but mod {n, f{x)) equals -1-6. But then which 

is impossible. □ 

3.2 An Efficient Probable Prime Test for n = 3 mod 4 

As noted above, the best results in pseudoprimality testing can be achieved when 
combining a Fermat- with a QF-based (or, analogously, a Lucas-based [14]) test 
for e(n) = —1 (cf. [2,5]). 

The arithmetic of the roots modulo p is intimately related to primality tests 
in terms of the Lucas U- and V- sequences which then yield the corresponding 
pseudoprimes. In this vein, n is defined a Lpsp{P, Q), ELpsp{P, Q), sLpsp{P, Q), 
respectively, exactly in the same manner as n is denoted a psp{a), Epsp{a) and 
spsp{a), respectively, where the conditions of the power polynomials are replaced 
by the corresponding ones of the Lucas sequences (cf. [14]). For simplicity we 
use the above abbreviations to denote Lucas pseudoprimes, Euler Lucas pseu- 
doprimes, strong Lucas pseudoprimes with respect to P, Q respectively, as well 
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as pseudoprimes, Euler pseudoprimes, strong pseudoprimes w.r.t. the base a, 
respectively (cf. [6,14]). 

The characterisation of the pseudoprimes in the last section which is based on 
the square root algorithms above, suggest the following probable prime testing 
algorithm. 

Firstly, suppose that n = 3 mod 4. 

1. Select randomly integers P G Z„,Q G Z* with = 1, = 

e(n) = -1. 

2. Let a = itQ~^ mod n. 

3 . [Is a a square root of Q modulo n?[ 

If = Q mod n go to step 4, else return ‘n is composite’. 

4. If x~^ ^ ^ Q (mod n, — Pa; + Q) return ‘n is composite’, 
else return 'n is a probable prime’. 

Theorem 2. Suppose that a composite integer n passes the test of this section. 
Then n is simultaneously a spsp{Q), a QFpsp{Q), and an ELpsp{P,Q) with 

e{n) = —1. Moreover, = 1 for all prime divisors p of n. 

Proof. The first assertion follows from above since any Epsp{Q) with = 
1 mod 2 already is a strong pseudoprime. But then V 2 (n — 1) > V 2 {ordn{Q)) = 
V 2 {ordp{Q)) for all p\n, where V 2 {c) denotes the highest power of 2 in the prime 

decomposition of c. Therefore = — 1 is impossible since otherwise V 2 ip—f) = 

V 2 {ordp{Q)) < V 2 {n — 1) = 1 which cannot be. Further, the general restricted 
period of Lemma 4 satisfies the conditions of an ELpsp{P, Q) since = 1. 

□ 

Remark 1. The approach of selecting the same base Q in the Fermat test as well 
as in the Lucas test (as second parameter) was first applied in [11] where also 
a fast method for finding P, Q G {2,-2} with (-^) = —1 is proposed when 
n ^ 1 mod 24 . Based on this parameter selection, an exhaustive search up to 
10^^ has yielded no composite that is simultaneously apsp{Q) and a Lpsp{P, Q). 

Remark 2. Note that in our test above, as also in the test described below, we 
require parameters P and Q such that both = 1 and (-^) = —1. For 

practical reasons, it is essential to be able to find these very rapidly. Although 
no formula is known that on input n, where n is an arbitrary integer, returns P 
and Q with the above properties, random search is usually very effective. Indeed, 
in [5] it is shown that the probability of failing to find such a pair is less than 
(3/4)®, where B = 50000 is the limit in the trial division step (which normally 
is performed before the actual pseudoprimality test). 

Although any pseudoprime to the above test has to fulfill the very strong 
properties above, it will turn out that the square root finding approach will 
yield an even more stringent test when n = 1 mod 4. This is analogue to the fact 
that any Epsp{a) for n = 3 mod 4 is already a spsp{a). 
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3.3 An Improved Probable Prime Test for n = 1 mod 4 

1. Select randomly P G Z„,Q in Z* such that = 1, = —1. 

2. Let b = mod (n, x^ — Px + Q) . 

3 . [Is b a square root of Q modulo n?[ 

If 5 ^ Z* or if 6^^Qmodn, return ‘n is composite’. 

4 . [ correct sign ? ] 

If ( ^ ) 7 ^ ^ return ‘n is composite’, otherwise return 
'n is a probable prime’. 

Remark 3. If both, a(P, Q)^ = a mod n and ( ) = !> trivially the condi- 
tion a{P, Q)^~ = ( ) o. mod n is fulfilled. Then, for the root —a mod n of 
Q, necessarily ( ) = —1 since (-^) = —1. But then also a{P,Q)^ = a = 
( ^ ) (~®) iiiod n, which forces n to fulfill the generalised Euler criterion. 

Example 1. As an example of a composite integer that satisfies for both roots 
a, a the original Euler-Lucas condition a(P,Q)^~ = a(P,Q)^~ mod n, but 
that does not pass the generalised Euler condition, we have n = 341 = 11 • 31, 
P = 2,Q = 4, because Un+i (P. O) = 0 modp, but = —1, where b = 

Vnii (P, Q ) /2 = 339 mod m 

Definition and remark: For any Q = (±a)^ mod n, because a does not appear in 
the left hand side of a(P, Q)^ = ±a mod n, the sign on the right hand side can 
only depend on P. We will throughout denote a = a{P) the sign corresponding 
to the square root of the Q ’s that occurs on the right hand side. That is, w.l.o.g. 
we take a = 1 for all those P corresponding to the smaller square roots of the 
Q’s on the right hand side, and <j = —1 for those P corresponding to the larger 
square roots. 

Theorem 3. Suppose there is a composite integer n that passes the above test. 
Then n is simultaneously an Epsp{Q), a QFpsp{Q) and a sLpsp{P,Q) for 

(■^) = —1. Moreover, = 1 for all prime divisors p of n. 

Proof. Similarly as above, since n is an ELpsp(P, Q) where now ” = 1 mod 

2, we conclude that n is already a sLpsp[P, Qf Clearly n fulfills E n+i (P,Q) = 

0 mod n. In particular, hM is odd. If we suppose that = —1 for some 

prime p dividing n, then Theorem 1 of [12] asserts that U n+i ^ Omodp, a 
contradiction. □ 

Recall that our pseudoprimality test was motivated by the square root finding 
problem of any residue Q. Actually, Grantham’s test is based on different ideas, 
as shortly indicated in the introduction (cf. also [5]). In short, his test consists 
of the following, where P and Q are chosen as above: 
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(1) Perform trial division by primes up to min {B, ^/n}, where B = 50000. 
In case of divisibility by one of these primes, declare n to be composite 
and stop. 

(2) If ^/n G Z declare n to be composite and stop. 

(3) Test if (mod n, — Px + Q) G Z„. 

(4) Test if = Q (mod n, x"^ — Px + Q). 

(5) Let — 1 = 2’’s with s odd. Test if = 1 (mod n, — Px + Q), or 
x^^^ = —1 (mod n, x^ — Px + Q) for some 0 < j < r — 2. 

When some of the steps (3)-(5) return a negative answer then n is composite, 
otherwise n is declared a probable prime. 

Remark 4- ^ Note that our test of this section fulfills steps (2) - (4) of Grant- 

ham’s. However, our conditions are at least as strong as his, since we incor- 
porate the actual square root b in the testing function, which additionally 
limits the chance for a composite number to pass (cf. also Lemma 5 below) . 

— To obtain a test that is in each of Grantham’s steps stronger than his, we 
only need to verify that n is additionally a strong probable prime to base b. 
Observe that if n passes, then it is already a probable prime to base b. 

— Grantham’s step (5) involves the odd parts of — 1. Gonsequently, when n = 
1 mod 4, our condition x~^ = b mod (n, x^ — Px + Q) implies Grantham’s 
condition, when n is a strong probable prime to base b. Unfortunately, we 
cannot extend this idea when n = 3 mod 4, as then is even. This explains 
why below only our test for n = 1 mod 4 will be further analysed. 

From the above we immediately get the following. 

Theorem 4. Suppose there is a composite integer n that passes the proposed 
test of this section and that is also a spsp{b). Then n passes also the probable 
prime test of [5]. 

4 Analysis of the Proposed Test for n = 1 mod 4 

4.1 A Sharpened Bound on the Error Probability 

In the following, we prove an even smaller bound on the probability that a 
composite number n is wrongfully identified as prime by our proposed test. 

As for Grantham’s test, we include trial division of small primes up to 
min {B, \/ri), where we choose B = 50000, as in [5]. Also, we test, if ^/n G Z. 
For convenience, we will call the test of Theorem 4 in combination with these 
two steps the multiplier dependent quadratic field based test (MQFT). 

Proposition 1. Let n be an odd integer. If p is a prime such that p^ divides n, 
then n passes the MQFT with probability less than ‘Ijp. 

Proof. We follow the proof of [5], but will get a better bound by a factor of 1/2. 
Let k be such that p^|n, but / n. 
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We will show in Lemma 5 below, there are at most parameters P with 
= — 1 that pass the test modulo for some Q w.r.t. a fixed a € 
{1,-1}. Similarly, it follows from Theorem 1 of [12] that there are at most 

of parameters P with ^ some Q. Each parameter P 

mod corresponds to {n/p^') parameters mod n, which gives less than ^ 
parameters P mod n for the fixed a. Analogoulsy we have the same bound for 
—a. So, in total there are at most (p — 1) • ^ parameters P for which there exists 

a Q, such that the pair (P,Q) passes. Since = 1 there are at most 

parameters Q modulo p^, and less than ^- 75 — • parameters Q mod n. 

z p 

k 1 fc 2 2 

Hence, altogether n passes for at most - — ^ = 5(1 — pairs 

(P, Q). The rest now follows exactly in the same manner as in [5]. □ 

Proposition 2. Let n be an odd eomposite with p\n. There are at most 
pairs (P, Q) mod p with ^ MQFT. 

Proof. This corresponds to Lemma 2.8 of [5]. The result follows exactly in the 
same way as in [5] since our test is at least as strong as Grantham’s. □ 

Consequently, we also obtain Lemma 2.9 of [5] concerning the number of 
pairs for which ~ ^ least one prime factor of n. Similarly, we adopt 

Corollary 2.10 and Lemma 2.11 of [5] in its direct form. 

Lemma 5. Let n = 1 mod 4 be any composite integer that passes the test of 
Theorem 4 for some elements P and some fixed Q = Qo with (±a)^ = Qo mod n. 
Denote a = a{P) any root of x"^ — Px + Qo modulo n and let p^ be any prime, 
respectively prime power, dividing n. Moreover, let a & { — 1,T\ be fixed. Then 

there are at most elements P with for which a{P, Q)^~ = 

a a mod p^. 

Proof. Suppose n passes the test for some P and the fixed Qq. Then f/n+i = 
0 mod n for all these P. As is odd. Theorem 1 of [12] implies that there are 
at most (^^,p— ~ 1 ~ 1 zeros P of Un+i = 0 modp^ 

since = 1 - 

We have to calculate the number of zeros that additionally fulfill a~^ = 
( ^ ® ^ ) (~®) ^ since = “1- Denote aa this fixed 

quantity modulo n. 

Clearly, the number of all zeros mod p^ is at most the number of those with 
the desired general multiplier. We now show that there are as many zeros P that 
fulfill a~^ = a a mod p^, as zeros that fulfill a^~ = —aa mod p^. 

It is known that if p{P) = p(p, P, Qo) is the rank of appearance mod p^ (cf. 
[4,15]) then there is a unique element s such that = s modp^. 
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where a denotes the conjugate root of a. We now write = p{P)t{P) for each 
of the zeros P. Then = era mod Since is odd and a 

multiple of p{P) for any zero P, both p{P) and t{P) need to be odd. It therefore 
suffices to show that the number of P with odd rank p{P) = r and multiplier s is 
the same as the number of the P' with same odd rank p(P') = r and multiplier 
—s. 

Recall that a = a{P) is uniquely determined by P. Now take any zero P with 
rank r and multiplier s. Then a{—P) = —a{P) modp^, and thus, a{—Py = 
{—lYa{Py = —s modp^ because the rank r is odd. Since p{P) = p{—P), also 
t{P) = t{—P) = t. Consequently, for any P with a{P)~^ = a{Py^ = s* = 
(TO mod we have that a{—P)~^ = (— s)* = —era mod p^, which proves the 
desired assertion. □ 

Remark 5. Observe the role of the a of the Lemma. As the parameters P are 
being varied, cr can take both values {1, —1}. But then, for each Q, there can only 
be two classes of P’s. Those with a(P, Q)^ = a, and those with a(P, Q)^ = 
—a mod n. 

The importance of the lemma lies in the fact that w.r.t. any Q = a? mod n, 
the quantity a limits the number of P modulo n that can pass the test. In 
detail, n passes condition 4 of the MQFT only for those P mod n for which 
= era mod pi for the same constant value cr € {1, —1} for all Pi\n. 

Lemma 6. Ifn=l mod 4 is squarefree and has k prime factors where k is odd, 
n passes the MQFT with probability less than 23 I -2 + 52 ■ 

Proof. The quantity -Jj- is the same as in Grantham’s proof for the number of 
pairs with = 1 for some prime p|n. In the following we can assume that 

= —1 for all p|n. 

We firstly develop an upper bound for the number of Q's for which there 
exists at least some P such that (P, Q) passes the test. Recall that necessarily 

= 1. So, for each p, the number of Q mod p is at most 

Suppose that for all the Q with = 1 for all p|n there exist some param- 
eters P = P{Q) mod n that pass the test. Then, 

a{P,Q)~^ = a mod n, ( 2 ) 

and, moreover, ( ) a = aa mod n has to be a correct square root of Q mod n. 
Further, the proof to Lemma 5 shows that w.r.t. — P mod n the root —aa mod 

n will be a correct square root. Since n = 1 mod 4, (— = 

(P) = — 1 , so that (— P, Q) will pass the first four steps for the general multiplier 
—aa mod n. 

Let now p be any prime dividing n. Then we have a(P, Q)~^ = a mod p and 
a(— P, Q)~^ = —a mod p (resp. for the conversed signs on the right hand side). 
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By hypothesis, this holds for every residue Q mod p, where the ±a mod p denote 
the square roots of these Q. Thus, by the assumption that the test passes the 
first four steps for some parameters P for each of the above Q, we obtain 
incongruent elements a mod p, and also incongruent elements —a mod p as 
images on the right hand side in (2), when reduced mod p. All these incongruent 
0,-0 mod p obviously comprise all 4>{p) elements in Z*. 

By assumption that for all the above Q there exist some P that pass the first 
four steps of the test, we can argue in the same way. For each prime q\n we again 
obtain 4>{q) incongruent square roots ±a mod q on the right hand side in (2) for 
q. In total, we obtain Hpin ~ 4‘{'n) incongruent square roots mod n of all 
the above Q. Now notice that each of these square roots has to be a basis for 
the strong probable prime test. However, the number of these ‘liars’ is at most 
(j){n)/A. 

Consequently, not all the Q’s with = 1 for all p\n pass the MQFT. 

We further investigate the set of the Q’s for which there exist P such that the 
test does pass. If we denote Sp this set reduced modulo some prime p\n, then Sp 
obviously is a subgroup of the group of the residues modp. This follows since 
all elements are squares, the above square root finding function is multiplicative, 
and the square roots of ±1 tested by the strong probable prime test, are trivially 
fulfilled modulo a prime p. 

By the Chinese Remainder Theorem, the above deduction shows that there 
has to be at least one prime pi\n for which the set Sp^ has cardinality less than 
Consequently, for this pi, Sp^ has at most ^ elements. 

On the other hand. Lemma 5 asserts that for each of the Q's there are less 
than < I parameters P that fulfill a~^ = a a modp for a fixed a mod n. 
Thus, by the Chinese Remainder Theorem, there are less than (pi/4) Y\p^p-^ f ' 

rip|„4 = 2 TOTT pairs for a fixed a mod n. Finally, for both a = 1 and —1, the 

2 

number of pairs that pass the MQFT is less than ^ . 

The desired probability now follows from the number of possible pairs (P, Q) 
which is at least \ (cf. [5]). 

□ 



We thus have our main results. 

Theorem 5. An odd composite number n = 1 mod 4 passes the MQFT with 
probability less than 

Proof. If n is not squarefree. Proposition I gives the result. If a squarefree n has 
an even number of prime factors we can apply Corollary 2.10 of [5], which gives a 
probability less than 2/B. Also, if n is squarefree and has exactly 3 prime factors, 
we use Lemma 2.11 of [5] which bounds the probability by -^ + 2 (b^- 3 b^) ■ 

In the remaining cases we apply Lemma 6 which gives largest probability 
when fc = 5 in which case it is bounded by 1/2^^ + 1/25000^. □ 

Theorem 6. Suppose that an odd composite integer n=l mod 4 is not one of 
the following: 
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— a product of exactly 5 prime factors, 

— a superstrong Dickson pseudoprime of type II (cf. [13]), i.e. one for which 
p^ — l\n — p for all prime factors p of n. 

Then n passes the MQFT with probability less than 25^- 

Proof. As above we distinguish the cases, n not squarefree, n squarefree and 
divisible by an even number of primes, n squarefree and divisible by three, re- 
spectively an odd number, of prime factors. We only need to improve on the 
bound of Lemma 5 as the remaining bounds are tight enough. We can assume 

that = —1 for all p\n. Any n that has exactly 5 prime factors yields the 

bound of Theorem 5, but for a larger number of prime divisors. Proposition 1 
and Lemma 5 yield the bound of this Theorem. This proves the first assertion. 

Now consider the second case. Then, for any p\n, = Q mod p 

and since Q is invertible, = 1 mod p. There are {n — p,p^ — 1) solutions of 
x^-P = 1 mod (p, — Px + Q). Each pair (P, Q) corresponds to two solutions 

with minimal polynomial x'^ — Px + Q which either both do or both do not 
satisfy a;”“P = 1. Thus, there are at most ^'^~P'P such pairs mod p. 

As in the proof of Lemma 6, Lemma 5 asserts that for any fixed a mod n we can 
only count half of the parameters P (for some Q) modulo each p, so that the 
number becomes at most — . Moreover, we also can count only the Q's that 
are squares modulo p, where as in the proof to Lemma 6 for at least one prime 

Pi|n the set of the Q’s has cardinality less than pi/4. Consequently, for p yf pi 
2 2 

there are less than and for pi less than pairs (P, Q) that can pass the 
test w.r.t. a fixed sign a mod n, corresponding to the signs of the square roots 
of the Q's. Then multiplying over all primes and adding for the two cr’s gives 
the desired probability. □ 

Remark 6. It is widely believed that type II superstrong Dickson pseudoprimes 
actually cannot exist. However, proving or disproving this claim has turned out 
to be quite delicate. Theorem 6 cannot be applied when such a number with 
exactly 5 prime factors does exist. This seems to be extremely unlikely. 



4.2 Peformance and Practical Considerations 

Although the MQFT w.r.t. the pair (P, Q) is defined in the context of the field 
Fp2, it can be rephrased in terms of Lucas sequences, as described below. 

Practically, the approach of utilizing Lucas sequences can be realised very 
fast. In particular, we only require the combined evaluation of both the U- and 
the V- sequence of same degree. But this can be obtained in almost the same time 
as the evaluation of the U- sequence by itself (cf. e.g. [17]). Moreover, the main 
advantage (as opposed to the second method below) is that many fast software 
packages for the rapid evaluation of Lucas sequences are readily available. 

1. Perform trial division by primes up to min {B , ^/n\ , as above. 

2. If ^/n G Z declare n to be composite and stop. 

3. If U n+i ( P. O) ^ 0 mod n declare n to be composite and stop. 
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4 . Let b = (^Vn+i (P, Q)j /2 mod n. If ^ Q mod n then declare n to 
be composite and stop. 

5. If ( ^ ) 7^ 1 declare n to be composite and stop. 

6. If n is not a spsp{b) then n is composite, otherwise declare n 
to be a probable prime. 

Nonetheless, the test can be performed even faster by utilizing binary addi- 
tion/Lucas chains [5, 10]. In particular, Q) is transformed into a sequence 

with second parameter equal to 1, then the combined evaluation of Vm{P,Q) 
and Vm+i {P,Q) niay be computed modulo n using less than 2 Ig n multipli- 
cations mod n and Ig n additions mod n. Moreover, as for m = the term 
U 2 m{P, Q) mod n has to vanish, it follows that the time for performing the Lucas 
tests in the MQFT is about twice the time to do a strong probable prime test. 
We obtain the same upper bound for the running time as Grantham does. 

Lemma 7. The time to perform the MQFT is about three times the time to do 
a strong probable prime test. 

As in Grantham’s case, this demonstrates the high efficiency of the MQFT. If 
one does three iterations of the strong probable prime test, a composite number 
will fail to be recognised as such at most 1/64 of the time. By contrast, in 
about the same time, the MQFT recognises composites with failure at most 
8190 (respectively 25000). 

Moreover, although spsp's to quite a large number of different bases are 
known [1,7] no composite number that passes Grantham’s test, and thus our 
proposed test, when n = 1 mod 4, has been found yet. Both these tests are, apart 
from their random choice of the parameters, refinements of the test by Baillie 
and Wagstaff [2]. However, the latter has proved to be extremely powerful in 
that nobody has yet claimed the 620 prize that is offered for a composite that 
passes it. 

Indeed, due to the existence of such strong tests, one might wonder, whether 
new pseudoprimality tests can be of any interest. We stress that in our case, the 
main motivation originally was of a different nature. It was actually the ques- 
tion of establishing an analogue of Euler’s criterion in quadratic extensions. We 
believe that the result that we found (Theorem 1) is of interest by itself. Addi- 
tionally, it then can be applied as an efficient pseudoprimality testing condition. 
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Abstract. In this paper, we propose three ideas to speed up the com- 
putation of the group operation in the Jacobian of a hyperelliptic curve: 

1. Division of polynomials without inversions in the base field, and an 
extended gcd algorithm which uses only one inversion in the base field. 

2. The omission of superfluous calculations in the reduction part. 

3. Expressing points on the Jacobian in a slightly different form. 



1 Introduction 

Using the Jacobian of a hyperelliptic curve defined over a finite field for appli- 
cations to cryptology was first proposed by Koblitz [6] . Practical algorithms for 
the group operation in such Jacobians have been proposed by Cantor [3] and 
Koblitz [7], [8]. Actual computations were done by Sakai, Ishizuka, Sakurai [12], 
and others. In the case of an elliptic curve, an algorithm for speeding up the 
group operation computations was proposed by Miyaji, Ono and Cohen [10]. 

The present paper discusses three ideas to speed up the computation of the 
group operation in the Jacobian of a hyperelliptic curve. The improvement is 
confirmed by counting the number of operations in the base field used. 

The time needed for a multiplications and b inversions in the base field we 
denote by aM+bl (additions in the base field are ignored). Although it is believed 
that / = (lO-l-e)M, for actual processors it is reported that I = {20 + e)M (Dec 
Alpha processor) and I = (30 -I- e)M (Intel Pentium processor); see Futa [5]. 

2 Cantor’s Algorithm 

In this section, we review the computation of the group operation in the Jacobian 
of a hyperelliptic curve as proposed by Cantor [3] . 

Let K he & perfect field with char (AT) ^ 2, and suppose C is a hyperelliptic 
curve of genus g, defined over K, given by an equation = F{x) for some 
F e K\x] of degree 2g + 1. Its Jacobian (more precisely, the K-rational points 
of it) can be regarded as the set 

J{C){K) := {(a, b) G A^(A'[a;]) | amonic, deg b < deg a < g, F — b^ = 0 mod a} . 
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The group structure on J(C){K) has ( 1 , 0 ) as unit element, and the inverse 
of (a, 5) is (a,—b). Let Pi = (ai{x) , bi{x)) G J{C){K) be two points. Cantor’s 
algorithm for computing Pi + P2 = P3 = (03(3;), 63(0;)) requires the next two 
parts. 

(Ideal Composition Part) 

1 - 1 . Take ei, 62, di such that di = gcd(oi, a2)(monic) and ciOi -I- 6202 = di. 

1 - 2 . Take ci, C2, d such that d = gcd(c?i, 6i-|-62)(monic) and cidi+C2{bi+b2) = d. 
1 - 3 . Put Si = cici, S2 = C2C2, S3 = C2, then we have d = siai-|-S2a2-l-S3(6i-|-62). 

1 - 4 . Put a = a\a2/d? and b = {siaib2 + S2a2bi + S3{bib2 + F))/d mod a. 

(Reduction Part) 

2 - 1 . Put a' = {F- b^)ja and b' = {-b mod a'). 

2 - 2 . If deg a' > g then a ^ monic(a'), b ^ b', goto 2 - 1 . 

2 - 3 . Put 03 ^ monic(a'), 63 ^ b' and output (03(3;), 63(3;)). 



3 Definition of Type I and Type II 

Definition 1 (Type I). The addition P\ + P2, (where Pi = (ai{x) , bi{x)) 
(i = l, 2 j^, is called an addition of type I, if the conditions gcd(ai,02) = 1 
and deg(oi) = deg(o2) = g are satisfied. 

In this case, the algorithm of Cantor can be simplified as follows. 

The Simplified Addition Algorithm for Type I 

I-l. Take ei, 62 such that ciOi -I- 6202 = 1 . 

1 - 2 . Put a = 0102 and b = 610162 -I- 620261 mod o. 

1 - 3 . Put 03 = (6^ — F)/a. 

1 - 4 . 03 ^ monic(o3) and 63 = —6 mod 03. 

I - 5 . If deg > g then 0^03, 6 ^ 63, goto 1 - 3 , else output (03, 63). 

Definition 2 (Type II). The duplication 2 Pi, (where P\ = {ai{x),b\{x))), is 
called a duplication of type II, if the conditions gcd(oi,6i) = 1 and deg(oi) = g 
are satisfied. 

Also in this case, the algorithm of Cantor can be simplified: 

The Simplified Duplication Algorithm for Type II 

II- l. Take ci, 62 satisfying eiOi -I- 20261 = 1 . 

II- 2 . Put o = Oi and 6 = eiOi6i -I- 62(61 -I- F) mod o. 

II- 3 . Put 03 = (6^ — F)/a. 

II- 4 . 03 ^ monic(o3) and 63 = —6 mod 03. 

II- 5 . If deg 03 > g then o ^ 03, 6 ^ 63, goto II- 3 , else output (03, 63). 

Lemma 1. Assume that the base field is K = Fp. Let Pi,P2 (resp. Pi) be (a) 
random point(s) of J(C)(Fp). The probability that the addition Pi + P2 (resp. 
the duplication 2 Pi) is not of type I (resp. type II) is 0 (i). 
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Proof. We only prove this for duplications and type II, since the remaining case 
is analogous. Let {a{x),b{x)) be a point of J(C')(Fp) which does not give a 
duplication of type II. Since ffJ{C)(¥p) is roughly p® and the number of monic 
polynomials whose degree is less than g is it follows that the probability 
that deg(a(a;)) < 5 is only 0{~). Hence we may and will assume that deg(a(a;)) = 
g and gcd(a(a;), 6 (a;)) yf 1. The condition a{x)\b‘^{x) — F{x) now implies that a 
monic h{x) G Fp[a;] exists such that h{x)\a{x), 0 < deg(h(a;)) < g, and h{x)\F{x). 

Let F[{x) be any monic polynomial in Fp[a;] with 0 < m := deg(iL(a;)) < g 
and F[{x)\F{X). Then 

ff{a{x) G Fp[a;] | a{x) monic, deg(a(a;)) = g, F[{x)\a{x) } = 

#{{a{x),b{x)) G J(C')(Fp) I deg a(a;) = g, H{x)\a{x) } < , and 

& JiC){¥p)\Hix)Hx)} ^ 1 

#J(C)(Fp) - 

The number of possibilities for F[{x) is finite and independent of p, so we obtain 
the desired result. 

The lemma shows that almost all additions and duplications satisfy the con- 
dition of type I and type II, respectively. Therefore, we will only discuss these 
cases in the following. 

Lemma 2. In the computation of an addition or duplication of type I and type 
II respectively, the number of loops in the simplified algorithm is at most 
where [r] denotes the largest integer < r. 

Proof. Initially one has deg(a 3 ) < 2g — 2. If deg(o 3 ) > 5 -I- 2 then deg(o 3 ) 
decreases by at least two in every cycle. In case deg(o 3 ) = p -I- I only one more 
cycle is necessary. This implies the result. 



4 Division of Polynomials without Using Inversions 



Consider the division with remainder f(x) = g{x) ■ s(x) + r(x) of a polynomial 
f(x) by a monic polynomial g(x). Here s(x) and r(x) are easily computed using 
synthetic division. However in case g(x) is not monic this computation requires 
inversion in the base field, which is time consuming. 

We propose two algorithms for the division of polynomials, which compute 
polynomials S(x) and R(x) and a constant D such that 



f(x) = g(x) ■ -b 



R{x) 

D ■ 



These algorithms do not involve any inversions. Write 



f{x) = '^Qixf g{x) = '^bixf S{x) = ^ ax\ R{x) 

i—0 i—0 i—0 



m— 1 

dix\ 

2=0 
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We assume n = deg f{x) > deg g{x) = m. 

Algorithm 1 (Faster when m) 

1. /3 <— bm, make a table of the /3* (z = 0, . . . , n). 

2 . (z = 0 ,...,n-l). 

3. Make a table of the (z = 1, . . . , to). 

4. Loop: j moves from 1 to zz — to + 1. 

4-1. Put r = Cn—m+l—j ^ kFn+1— J 

4-2. Put Wn+l-i-j = Wn+l-i-j - r(3'‘~^bra-i (z = 1, . . .,to). 
(End of loop) 

5. Take di = WiP^ (z = 0, . . . , to - 1). 

6 . Put Ci ^ (z = 0, . . . , rz — to). 

7. Take D = /3". 

Algorithm 2 (Faster when n = m) 

1. P ‘^bm, Wi^ ai (z = 0 , . . . , rz - 1). 

2. Make a table of the /3* (z = 1, . . . , rz — to -I- 1). 

3. Loop: j moves from 1 to rz — to -I- 1. 

3-1. Put T = Cn—m+l—j ^ kFn+1— j 

3-2. Put — — P^^n+l — i—j '^bm—i ip — I5 ■ ■ ■ 5 to). 

3-3. If rz — TO — j > 0 then Wn-m-j = Wn-m-jP^ ■ 

(End of loop) 

4. Take di = Wi (z = 0, . . . , to — 1). 

5. Put Ci ^ CiP'^ (z = 0, . . . , rz — to). 

6. Take D = 



5 Extended Gcd with Only One Inversion 



In general the Euclidean algorithm (cf. [4]) is used for an extended gcd calcu- 
lation, which means computing ei(a;), 62(3;), and a monic gcd(/(a;), g(a;)) such 
that ei(a;)/(a;) -I- e 2 (x)g(x) = gcd(f(x),g(x)) for given polynomials f(x), g(x). 
We will apply our division of polynomials to this. Assume degf(x) > degg(x). 

Improved Euclidean Algorithm 

1- /o(a;) ^ f(x), go(x) ^ g(x), i = 0. 

2. Apply division with remainder to obtain Si{x), Rpx) and a constant Di such 
that fi{x) = gi{x) ■ Si{x)/Di + Ri{x)/Di. 

3. If Ri(x) = 0 then z ^ z — 1, goto 7. 

5. If deggi+i(x) = 0 then goto 7. 

6. z ^ z -I- 1, goto 2 . 



now 



fi+l 

9i+l 



= Mi-... -M, 



X X 

ei(a;) 62(3;) 



— Mi ■ Mi-i ■ ■ ■ Mq 



7 . If z > 0 then compute 61(3;), 62(3;) by 

(and if z = — 1 then 61(3;) ^ 0, 62(3;) ^ 1). 

8. Put a as leading coefficient of gi+i{x). 

9. Fnt gcd{f{x),g{x)) ^ a~^gi+i{x), ci{x) ^ a~'^ci{x), 62 ( 3 ;) 



^62(3;). 
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The algorithm above computes only once inversion Hence it is faster 

than the original Euclidean algorithm, if inversion in the base field is much slower 
than multiplication. This is the case, for example, if the base field is Fp for a 
large prime p. Since the extended gcd algorithm is used in the algorithms for 
computing in the group J(C')(Fp), these will perform faster as well. 

6 Improvement of the Reduction Part 

In the computation of the reduction part, the division a' = {b^ — F) /a turns 
out to be a bottleneck step. The fact that this is a division with remainder 0, 
allows one to omit some calculations of low degree terms here. More precisely, 
put n = deg a, then the following holds. 

1. In the computation of 6^, only the terms with degree > n are needed. 

2. To compute (6^ — F)/a, only terms of — F with degree > n are needed. 
These observations allow one to make the we computation of (6^ — F) /a three 
to four times as fast. 

7 Expressing Points on J(C){K) Using a Denominator 

In section 5, we propose a improved extended gcd algorithm which computes 
ei{x)f{x) + e 2 {x)g{x) = gcd{f{x),g{x)). Since we required the gcd to be monic, 
this algorithm needs one inversion in the base field. If we allow the gcd to be 
a non-monic polynomial, this inversion is not needed anymore. In this section, 
we attempt to use an extended gcd algorithm which outputs as gcd a possibly 
non-monic polynomial, to the computation of the group law in J(C){K). For 
this purpose, the second coordinate of a point in J{C){K) will be expressed as 
b{x) = B{x)/l for some I G . 

The numerical experiments described in the next section suggest that if the 
genus g is an even number, this leads to an improvement. However, if the genus 
g is odd, the performance is not better (if g is odd, then a' obtained in the last 
cycle of the reduction part must be monic, and some calculations are omitted.) 

We now present the algorithms for the group operations of type I and type 
II, improved using sections 5, 6, and 7. Let Pi, P2 be points in J{C){K) written 
in the form Pi = {ai{x), Bi{x)/li) for i = 1,2. 

Algorithm for Type I (Computation of Pi + P2) 

I-I. Take Ei,E 2 ,d as Eiai + £^202 = d. (Remark that d G .) 

1-2. Put a = 0102 , B = EiaiB 2 h + £ 202 £ 1^2 mod o, I = lil 2 d. 

Here B is computed using the following steps: 

1-2-1. Compute £iOi. 

1-2-2. Compute (£iOi)(£ 2 ^i)- 

1-2-3. Compute {d — £iOi)(£i^ 2 ) which equals £ 202 £i^ 2 - 
1-2-4. Put B = EiaiB 2 h + £ 202 £i ^2 mod a. 

1-3 (Reduction Step) 

1-3-1 Compute P. 
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1-3-2 Compute the coefficients of PF whose degree > deg a. 

1-3-3 Compute the coefficients of whose degree > deg a. 

1-3-4 Put a' = {PF-B^)/a. 

1-3-5 Take a' ^ monic(a'). 

1-3-6 Put B' = -B mod o'. 

I- 3-7 If deg a' > g then a ^ a' , B ^ B' and goto 1-3-2, else output (a{x),B{x) /I). 

Algorithm of Type II (Computation of 2Pi) 

II- 1-1 Compute 2Bi (= Bi + Bi). 

II- 1-2. Take Ei,E 2 ,d as Ei{liai) + E2{2bi) = d. (Remark that d € .) 

11-2. Put a = a\, B = EiaiBip + E 2 B 1 + E 2 I 1 F mod a, I = l\d. 

Here B is computed using the following steps: 

II- 2-1. Compute E 2 B 1 . 

II- 2-2. Compute Bi{d — E 2 B 1 ) which equals to EiaiBip + E 2 BI. 

II- 2-3. Compute 11 and 7^2 • 

II-2-4. Put B = EiaiBili + E 2 B 1 + E 2 I 1 F mod a 
II-3 (Reduction Step) 

II-3-1 Compute P. 

II-3-2 Compute the coefficients of PF whose degree > deg a. 

II-3-3 Compute the coefficients of whose degree > deg a. 

II-3-4 Put a' = {PF - B'^)/a. 

II-3-5 Take a' ^ monic(a'). 

II-3-6 Put B' = -B mod a'. 

II-3-7 If deg a' > q then a ^ a' , B ^ B' and goto II-3-2, else output 
{a{x),B{x)/l). 

8 Numerical Experiment 

In this section, we evaluate the computational quantity aM + bl for our group 
law algorithms on J(C')(Fp). For this purpose, we make low level functions for 
inversion and multiplication modulo p, and count how often these functions are 
called. By convention, we do not count a trivial operation, for example, the cases 
ax0 = 0, 2x0 = (o-|-a) mod p, a/1 = a etc. All further programs for operations 
on polynomials, the extended gcd, and the group law algorithm use only these 
low level functions when dealing with multiplications and inversions modulo p. 

The experiments were done by taking various prime numbers p = 10®. The 
following tables show the maximal values of aM and bl over the chosen set of 
primes. We remark that in almost all cases the maximal values aM -\- bl, are not 
very different from the value for a fixed used prime p. 

In the tables the following notation is used. 

A: without any suggested improvements. 

B: using only the improved extended gcd discussed in § 5. 

C: using the improved extended gcd and the reduction part as in § 6. 

D: using all three improvements discussed in this paper. 
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Addition of Type I 

C : y'^ = + ... + + a; + 1 



g: genus 


A 


B 


C 


D 


2 


70 M+ 3 I 


71 M+ 2 I 


52 M+ 2 I 


55 M+ 1 I 


3 


200 M+ 4 I 


204 M+ 2 I 


144 M+ 2 I 


154 M+ 2 I 


4 


386 M+ 6 I 


398 M+ 3 I 


286 M+ 3 I 


289 M+ 2 I 


5 


694 M+ 7 I 


717 M+ 3 I 


496 M+ 3 I 


510 M+ 3 I 


6 


1054 M+ 9 I 


1091 M+ 4 I 


756 M+ 4 I 


759 M+ 3 I 


7 


1604 M+ 10 I 


1658 M+ 4 I 


1114 M+ 4 I 


1132 M+ 4 I 


8 


2186 M+ 12 I 


2260 M+ 5 I 


1516 M+ 5 I 


1519 M+ 4 I 


9 


3042 M+ 13 I 


3139 M+ 5 I 


2054 M+ 5 I 


2076 M+ 5 I 


10 


3894 M+ 15 I 


4017 M+ 6 I 


2622 M+ 6 I 


2625 M+ 5 I 



Addition of Type I 

C : y'^ = a;^®+^ + A 2 gx'^^ + .... + Aia; + Aq Aiirandom numbers 



g: genus 


A 


B 


C 


D 


2 


70 M+ 3 I 


71 M+ 2 I 


52 M+ 2 I 


56 M+ 1 I 


3 


200 M+ 4 I 


204 M+ 2 I 


144 M+ 2 I 


157 M+ 2 I 


4 


386 M+ 6 I 


398 M+ 3 I 


286 M+ 3 I 


292 M+ 2 I 


5 


694 M+ 7 I 


717 M+ 3 I 


496 M+ 3 I 


515 M+ 3 I 


6 


1054 M+ 9 I 


1091 M+ 4 I 


756 M+ 4 I 


764 M+ 3 I 


7 


1604 M+ 10 I 


1658 M+ 4 I 


1114 M+ 4 I 


1139 M+ 4 I 


8 


2186 M+ 12 I 


2260 M+ 5 I 


1516 M+ 5 I 


1526 M+ 4 I 


9 


3042 M+ 13 I 


3139 M+ 5 I 


2054 M+ 5 I 


2085 M+ 5 I 


10 


3894 M+ 15 I 


4017 M+ 6 I 


2622 M+ 6 I 


2634 M+ 5 I 



Duplication of Type II 

C : y'^ = + a;^® + ... + a;^ + a; + 1 



g: genus 


A 


B 


C 


D 


2 


66 M+ 3 I 


68 M+ 2 I 


49 M+ 2 I 


55 M+ 1 I 


3 


186 M+ 4 I 


192 M+ 2 I 


132 M+ 2 I 


146 M+ 2 I 


4 


359 M+ 6 I 


372 M+ 3 I 


260 M+ 3 I 


268 M+ 2 I 


5 


650 M+ 7 I 


673 M+ 3 I 


452 M+ 3 I 


472 M+ 3 I 


6 


989 M+ 9 I 


1025 M+ 4 I 


690 M+ 4 I 


700 M+ 3 I 


7 


1514 M+ 10 I 


1566 M+ 4 I 


1022 M+ 4 I 


1048 M+ 4 I 


8 


2067 M+ 12 I 


2138 M+ 5 I 


1394 M+ 5 I 


1406 M+ 4 I 


9 


2890 M+ 13 I 


2983 M+ 5 I 


1898 M+ 5 I 


1930 M+ 5 I 


10 


3705 M+ 15 I 


3823 M+ 6 I 


2428 M+ 6 I 


2442 M+ 5 I 





446 



Koh-ichi Nagao 



Duplication of Type II 

C : + .... + A\x + Atg for random numbers Ai 



g: genus 


A 


B 


C 


D 


2 


76 M-k 3 I 


78 M-k 2 I 


59 M-k 2 I 


66 M-k 1 I 


3 


207 M+ 4 I 


213 M-k 2 I 


153 M-k 2 I 


170 M-k 2 I 


4 


395 M+ 6 I 


408 M-k 3 I 


296 M-k 3 I 


307 M-k 2 I 


5 


705 M+ 7 I 


728 M-k 3 I 


507 M-k 3 I 


532 M-k 3 I 


6 


1067 M+ 9 I 


1103 M-k 4 I 


768 M-k 4 I 


783 M-k 3 I 


7 


1619 M+ 10 I 


1671 M-k 4 I 


1127 M-k 4 I 


1160 M-k 4 I 


8 


2203 M+ 12 I 


2274 M-k 5 I 


1530 M-k 5 I 


1549 M-k 4 I 


9 


3061 M+ 13 I 


3154 M-k 5 I 


2069 M-k 5 I 


2110 M-k 5 I 


10 


3915 M+ 15 I 


4033 M-k 6 I 


2638 M-k 6 I 


2661 M-k 5 I 



Appendix 



Dr. Y. Futa of Matsushita Electronics suggested that the improved extended 
gcd algorithm in section 5 may be applied to the computation of inversions in 
Fpn. Let f{x) = a;” + aa;”“^ + . . . + 6 G Fp[a;] be a monic irreducible polynomial. 
Then an element of Fpn = Fp[a;]/(/(a;)) is represented by a polynomial g{x) with 
degg(a;) < n. Computing the extended gcd ei(a;)/(a;) + e 2 {x)g{x) = 1 yields the 
inverse element g{x)~^ = e 2 {x). Futa also suggested that the time needed for 
this inversion is essentially less than that for the extended gcd, because the 
computation of ei(a;) can be omitted. 

Futa and I estimate that the above idea uses — 3)M + I. In the 

special case where f{x) is of the form a;” — w, even — 4)M + 1 suffices. 

When n is more than 6 or 7, this value may be smaller than the value used in 
Daily and Paar’s method [2], which is known as the fastest inversion method. 
It should be noted that our method does not need the condition on p called 
OEF [1]. 



Postscript 

Further actual computation of the Jacobian of a hyper elliptic curve defined over 
F 2 *» is done by Tamura and Sakurai [14]. The similar computation of extended 
gcd is also done by Lim and Hwang [9]. 
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Abstract. Using Weil explicit Formulas, we show how to compute the 
multiphcity of a zero at the point ^ of the Artin L-functions associated 
to a character x of Degree 2 in quaternion helds of degree 8. We prove 
in several examples that — 0 when lU(x) = 1 and — 1 when 
W{x) = -1. 



1 Basic Properties of Artin L-Functions and Conjectures 

Let N/K be a Galois extension of a number field with a group G — Gal{N/K) 
and let (p, V) be a representation of G and x denotes its character, then the 
Artin L-function attached to x is defined by: 

L(N/K, X, «) = n det(l-^qjA(p)-)’ 

where the product is over unramified primes of K and is the Frobenius 
automorphism . The Artin L-function converges uniformly in the half-plane 
die(s) > 1-|-J (J > 0) and defines an analytic function on the half plane die(s) > 1. 
From basic properties of Artin L-functions we have the identity: 

Theorem 1. 

We have: 

CN{.s)^CK{s)Y{L{N/K,xGr^^\ 

where x varies over the non trivial irredueible eharaeters of G and x(l) appear 
as the unique positive integers eoeffieients in the deeomposition of the regular 
representation reg^j of G (ef [1]) : reg^j = 

In order to obtain an L-function with a functional equation, it is necessary to 
introduce Euler factors at infinite primes of K . Let us define: 

A{N/K, X, s) = c{N/K, x)^L^{N/K, x, s)L{N/K, x, s), 

where 

c{N/K,x) = \dK\’^^AN{f{N/K,x}} 



W. Bosnia (Ed.): ANTS-IV, LNCS 1838, pp. 449-458, 2000. 
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and 

Loo ^\{L,{N/K,x,s). 

p|oo 

For every infinite place p of K , we put: 



where 



and 



Lp{N/K,x,s] 



if p is complex, 

Lk(s)"'^Lk(s + 1 )" if p is real. 



Lc{s) = 2{27r)-^' r{s), Lr.{s) = 

+ ^ x(i) + x(y’q?) ^ X(1) -x(y’y) 

2 ’ 2 



Using the Brauer induction theorem, we have the following result: 

Theorem 2. 

The completed Artin function A{N/K,x,s) has a meromorphic continuation to 
the complex plane and satisfies: 

A{N/K, X, s) = W{x)A{N/K, x,l-s) 

where W{x) of absolute value 1. 

The reader is referred to [2] for a good introduction to the Artin L-functions 
and character theory. 

Artin ’s conjecture asserts that for every irreducible character y 7 ^ 1, the Artin L- 
function L{N / K, y, s) has an analytic continuation. This means that the quotient 
^ is entire. Actually Aramata and Brauer proved the following theorem (cf. [3]): 

Theorem 3. 

If N / K is a Galois extension then the quotient ^ is entire. 

Now we restrict our attention to the multiplicity of a zero of an Artin L-function. 
In this direction Stark proved the following result (cf. [4]): 

Theorem 4. 

Let us denote n^(so) — ordg-sg L(N/K, x, s) and r — ord^^so Ca^(s), then we 
have the inequality: 

X irreducible 

In the particular case where sq = we denote n-^ — If we assume the 

Generalized Riemann Hypothesis (GRH), we can show the following estimate 
for n^: 



n 



X 



^ 3 ln|dAr| 
~ 2 Inin |djv| 



Theorem 5. 

lUe have: 
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Proof. 

To prove the inequality above, we use the Weil explicit formulas applied to Cn{s) 
(cf. [5]): 

Theorem 6. (Weil Explicit Formulas) 

Let F satisfy the eonditions (A) and (B) below and F{0) — 1: 

(A) F is eontinuous and eontinuously differentiable everywhere exeept at a finite 
number of points a{, where F{x) and F'{x) have only a diseontinuity of the first 
kind, sueh that F{ai) — ^{F{ai + 0) + F{ai — 0)). 

(B) There is a number b > 0 sueh that Fix) and F'{x) are as 

I X |— 7- oo. 

Then the Mellin transform of F : 




is holomorphie in every strip —a < cr < 1 + a where 0 < a < b, a < 1, and the 
sum running over the non trivial zeros p — /3 + ij of^j^fs) with | 7 |< T 

tends to a limit as T tends to infinity. This limit is given by the formula: 

( 1 ) Y.^{p) = ^( 0 ) +^( 1 ) - 2 ^ i^^^T(mlu(iV(p))) 

p p,m 

+ ln(|(iAr|) - n[ln(27r) + 7 + 21n(2)] - riJ{F) + nI{F) 



where 



J{F) 



F{x) 



r+00 

/o 2cosh(|) 



dx, I{F) 



o-\-00 



and 7 = 0.57721566 • • • is the Euler eonstant. 
Now let us dehne F by: 



1 - Fix) , 
2 ^mh(|)^^ 



Fix) 



1— I X 

0 



if I a: |< 1 
otherwise. 



then we have: 

Lemma 1. 

The Fourier transform of F is: 

Fiu) 



2sm(|)V 



Let us put Ft[x) — F{y), then Ft{u) — TF{Tu). If we apply theorem 6 to Ft, 
we obtain the inequality: 



n%T < 4 



cosh(— + ln(|djv|) + n 



'0 2Tsinh(|) 



dx. 



nxT < 62 + ln(|djv|) + n 
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we put T — 21iilii(|(ijv|) then: 

ln(|cfAr|) n 3 Jn(|rfjv|)_ 

~ lnln(|(ijv|) 21nln(|(ijv|) ~ 2 lnln(|(ijv|) ' 

□ 



Now we shall recall some important conjectures on and n^{so): 

Conjecture 1. 

In the case of the Dirichlet L series, we have — 0. 

Conjecture 2. 

Generally for every irreducible character y, we have: 

%(«o) <X(1) 

or even the stronger result: 

%(«o) < 1. 

2 Quaternion Extensions 

In this section we describe some properties related to the construction of quater- 
nion helds and their associated Art in L-functions. We shall restrict ourselves to 
the case of a tamely ramihed extension. 

Definition 1. 

A quaternion extension ofQ is a normal extension N of Q with Calois group G 
isomorphic to the quaternion group Hs of order 8. 

The quaternion group is the unique group of order 8 having 3 cyclic subgroups 
of order 4. 

One can write Hs —< <t,t > with relations — 1, and = (T“^, 

so there exists a unique irreducible character of degree 2 of Hs verifying y(l) = 2, 
y((7^) = —2 and y(s) — 0 fov s ^ 1, . 

N contains three quadratic subhelds ki, k 2 , ks with discriminant di, d' 2 , ds and 
a biquadratic subheld K with discriminant d\d 2 ds. One can show that N can 
be written N — K{\/M) for some M £ K such that trjf/((j(M) = ±_l±2i±Ai±d^ 
mod 4 (cf. [6]). The theorem below tells us under what condition a quadratic 
held k — Q(vd) can be embedded in a quaternion held N (cf. [7]). 

Theorem 7. 

Let d be a square free integer. In order that k — Q(Vd) is a quadratic subfield of 
some quaternion field N , it is necessary and sufficient that d be positive and not 
congruent to —1 mod 8. 

Example 1. 

ki — Q(V5), k 2 — Q(v^), ks — Q(Vl05) are quadratic subhelds of the quater- 
nion extension N — K{\fM) where M — One can verify that the 

extensions N/ki,N/k- 2 , N/ks are cyclic and so A is a quaternion extension of Q. 
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Example 2. 

One can check the same for k\ — Q(V5), ^2 = Q(v®) and N — K{\/M) where 
M - 5+V^ . 41+5V4l ^ 

In the last section we will give a table of many totally real and imaginary quater- 
nion extensions with their quadratic subhelds. 

Now we restrict our attention to the Artin L-function L{s,x) associated to the 
unique character y of degree 2 of H^. If we write L{s,x) in terms of Dedekind 
zeta functions, then by using theorem 1, we have: 

Proposition 1. 

IFe write: 

Cn{s) - C,K{s)L{s,xf ■ 

We know that is an entire function (theorem 3), so L‘^{s, x) is holomorphic 
on the whole plane. Since L{s,x) i® meromorphic (theorem 2) we deduce the 
following proposition: 

Proposition 2. 

The Artin L{s,x)-funetion is entire. 

We dehne an invariant f/jv of the quaternion extension N by: 



Un 



1 if the ring of integers Ojv of N is a free Z[G]-module, 
— 1 otherwise. 



The Frbhlich theorem gives the general equality (cf. [7], [8]): 

Theorem 8. 

IFe have: 

IF(y) = Fat. 



Let us denote: 

_ r 1 if N is real, 

( — 1 if N is imaginary. 

In [6], one can hnd an effective criterion to know if Ojv is a free Z[G]-module or 
not: 

Theorem 9. 

IFe have: 

On is a free '&[G\-module if and only if 

n _l + di + d2 + ds 

p — ^ mod 4. 

p\cIm 



A look at the functional equation of L{s,x) shows: 
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Theorem 10. 

IfW{x) — 1 then is even, 

IfW{x) = — 1 then n^ is odd. 

Conjeeture 3. 

If IT(x) = 1 then n^ — 0, 

If W{x) — —1 then n^ — 1 . 

In fact, conjecture 3 gives more information on n-^ than conjecture 2 insofar as 
it says that n-^ is the smallest possible with respect to constraints imposed by 
the sign of W{x) when y is real- valued. 

3 Computation of 

In this section we give an explicit method to compute n-^ and verify conjecture 3 
in many cases. If we suppose that Civ(|) 7 ^ 0 (conjecture 1) then 



2n-^ — ordi Cn{s). 



In practice for a given quaternion held N , we show that Civ(^) 7 ^ 0 and compute 
ordi Cn{s). Let us write the Weil explicit formulas for Cw, and let us consider 

Serre’s function Fy{x) — e~^^ (j/ > 0). The Mellin transform ^{s) of Fy is 



^s) 



7T 

g4yV 2/ 



and the Fourier transform Fy of Fy is 



Fy{t) = 



7T _ 

— e 

y 



If we assume GRH for Cw, we have d>{p) — Fy{t) where p — ^ it. For every 
k > 1, we denote by G the positive imaginary part of the zero of the Dedekind 
zeta function, and n^ its multiplicity. We have the identity: 



+ 0O ^2 

S{y) - % -F 

k>2 

- eW - , V ^^(^(P)) g-Gmln(JV(p)))^ 

^ p,m 



■ [ln(|dAr|) - 8 [ln( 2 ^) + 7 + 21 n( 2 )] - ri.J{Fy) + 8 /(F^)] , 



Here we have ri = 0 or ri = 8 . Now we need the following theorem: 
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Theorem 11. 

We have the inequality for every y > 0: 

Ux < S{y) 



and 



lim S(y) — n 

y^O 



We should notice that the advantage of Serre’s function is that the series S{y) 
converges quickly to nx when j/ — 0. In practice we prove in many quaternion 
helds that when W (y) = 1, we have nx < S{y) < 2 for some j/ > 0 and so = 0. 
Similarly for W{x) = then one can prove the inequality nx < S{y) < i for 
some y > 0 and so = 1. Now to prove that Civ(|) 7^ 0? we apply again the 
explicit formulas to (k («) with the same function Fy and show as we did before 
that 

ordi Ck{s) <S{y) <1 

for some y > 0 and thus we have ordi Civ(^) = 0. To compute S{y), we shall 
compute the integrals I{Fy) and J{Fy) with a given precision. The series over 
the prime ideals Vqo in. the Weil explicit formulas is truncated to 



wAv) = X) X) ln(iV(p)) 

mln(iV(rt)<cons 



y-y{m ln(N(p))f 



where cons = 

The condition mln(N(p)) < cons means that we don’t take into account the 
terms of the series less than 10“^’. In practice we take c = 30 and po less than 
10®. The number held being dehned by a polynomial P{x), for every prime 
number p prime to the index of the held, the decomposition of the ideal (p) 
into a product of prime ideals of the held is given by the decomposition of P{x) 
modulo p (cf. [9]). In the case where p divides the index, we use a stronger 
algorithm (see algorithm 6.2.5 in [9]). Actually in both cases, we don’t need to 
compute explicitly the factors of P{x) modulo p, we just need to compute the 
degree of each factor in order to compute the norm of the associated prime ideal. 
Since N/Q is a Galois extension, then one need to compute only the degree of 
the hrst irreducible polynomial appearing in the decomposition of P{x) modulo 
p. This allows us faster computations. In fact the experimental value of S{y) is 
S{y) > S{y) and so nx < S{y). One can prove the following estimate by using 
the prime number theorem: 

Theorem 12. 

If we take cons = oo, then the following estimate holds: 
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III the following computations, we give the reduced polynomial of the quater- 
nion held N/Q, the discriminant djv of N, two quadratic subhelds Q(Vdi) and 
Q(v^) of N ( the third one is in fact Qiy^dj ^ ) ), W{x) and n^. 

1) iV/Q: P{x) -f 29^?^^ -f - 3052 ?^ _ 1345a? - 395 

djv = 3® • 5® • 7® 

quadratic subfields: Q(V5) and Q(v^) 

Wix) = 1 

Ux = 0 . 

2) N/Q: P{x) = a?« -h 315a?® -f 34020a?4 -H 1488375a?2 -h 22325625 
djv = 3® • 5® • 7® 

quadratic subfields: Q(V5) and Q(v^) 

Wix) = -1 

Ux = 1 . 

3) N/Q: P{x) = a?« - 205a?® -f 13940a?4 - 378225a?2 -h 3404025 
dM = 5® -41® 

quadratic subfields: Q(V5) and Q(v®) 

Wix) = -1 

Ux = 1. 

4) N/Q: P{x) = a?« - 3a?^ -f 142a?® - 115a?® -f 6641a?4 -H 3055a?® -f 157938a?® -f 
152941a? -h 2031361 

djv = 3^ • 5® • 41® 

quadratic subfields: Q(V5) and Q(v®) 

Wix) = -1 

Ux = 1. 

5) N/Q: P{x) = a?« - a?^ - 178a?® - 550a?® -f 7225a?^ -H 44407a?® -f 55928a?® - 
45392a? -f 4096 

dA? = 3® • 11® • 17® 

quadratic subfields: Q(Vl7) and Q(v^) 

Wix) = 1 

Ux = 0. 

6) iV/Q:P(a?) = a?«-3a?^-hl06a?®-h381a?®-h414a?4-8475a?®-h44497a?®-hl51740a?-h 
253168 

dA? = 3® • 11® • 17® 

quadratic subfields: Q(Vl7) and Q(v^) 

Wix) = -1 

Ux = 1. 
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7) N/Q:P{x) = ir«-3ir^-475a?®-2386a?H56669a?^ + 732202ir3 + 3280440a?2 + 
5788174;^+ 2396941 

djv = 37® -41® 

quadratic subfields: (Q(v^) and (Q(v®) 

W{x) = -1 

Ux = 1 . 

8) N/Q:P{x) = a:«-3a:^-847a:®-4250a:®+194805a:4+2321042a:3+4218300a:2- 
28827252a: - 48031623 

djv = 37® • 73® 

quadratic subfields: (Q(V37) and (Q(v^) 

Wix) = -1 

Ux = 1 . 

9) N/Q: P{x) = a:« - 3a:^ + 1854a:® + 14657a:® + 1134753a:4 + 15385779a:® + 

370857442a:2 2861780247a: + 28470071727 

37® • 73® 

quadratic subfields: (Q(v^) and (Q(v^) 

W{x) = -1 

Ux = 1. 

10) N/Q: P{x) = a:« - 3a:^ + 1042a:® + 8233a:® + 284219a:4 + 4899401a:® + 
42209694a:® + 179998937a: + 404059099 

djv = 3^ • 37® • 41® 

quadratic subfields: (Q(V37) and (Q(v®) 

W{x) = -1 

Ux = 1. 

11) N/Q: P{x) = x^ - x^ - 866a:® - 2686a:® + 197617a:4 + 1072207a:® - 
8786448a:® - 32864208a: + 159160192 

■ 17® • 23® 

quadratic subfields: (Q(Vl7) and (Q(Vl61) 

W{x) = 1 

Ux = 0. 

12) N/Q: P{x) = a:® - 3a:^ - 1591a:® - 7978a:® + 718061a:4 + 8174530a:® - 
29006964a:® - 433628432a: + 235862473 

dN = 37® • 137® 

quadratic subfields: (Q(V37) and (Q(Vl37) 

W{x) = -1 

Ux = 1 . 

13) N/Q: P{x) = a:® - 3a:^ + 3478a:® + 27505a:® + 4489397a:4 + 53881703a:® + 
2972520282a:® + 26220344507a: + 651061429207 

djv = 3^ • 37® • 137® 

quadratic subfields: (Q(v^) and (Q(Vl37) 

W{x) = -1 

Ux = 1 . 
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Abstract. There are 38975 Fermat pseudoprimes (base 2) up to 10^^, 
101629 up to 10^^ and 264239 up to 10^®: we describe the calculations 
and give some statistics. The numbers were generated by a variety of 
strategies, the most important being a back-tracking search for possible 
prime factorisations, and the computations checked by a sieving tech- 
nique. 



1 Introduction 

A (Fermat) pseudoprime (base 2) is a, composite number N with the property 
that 2^“^ = 1 mod N. 

For background on pseudoprimes and primality tests in general we refer to 
Bressoud [1], Brillhart et al [2], Koblitz [4], Ribenboim [12] and [13] or Riesel 
[14]. Previous tables of pseudoprimes were computed by Pomerance, Selfridge 
and Wagstaff [11]. 

We have shown that there are 38975 pseudoprimes up to 10^^, 101629 up to 
10^^ and 264239 up to 10^^; all have at most 9 prime factors. Let P (A) denote 
the number of pseudoprimes less than X and let P {d, A) denote the number 
with exactly d prime factors. In Table 1 we give the values of P (A) and P (d. A) 
for d < 9 and A in powers of 10 up to 10^^. 

We began the computations described here some years ago and earlier ver- 
sions have already been cited in the literature. We therefore feel it appropriate 
to document the techniques used. The data files are available at 
ftp://ftp.dpmms.CELm.ac.uk/pub/PSP or from the author. 

The pseudoprimes were generated by a variety of strategies, the most im- 
portant being a back-tracking search for possible prime factorisations, and the 
computations checked by a sieving technique, together with a “large prime vari- 
ation” . 

We also used the same methods to calculate the smallest pseudoprimes with 
d prime factors for d up to 16. The results are given in Table 2. 



2 Some Properties of Pseudoprimes 

In this section we discuss some elementary properties of pseudoprimes and the 
overall search strategy. 



W. Bosnia (Ed.): ANTS-IV, LNCS 1838, pp. 459-473, 2000. 
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X 


d = 1 


2 


3 


4 


5 


6 


7 


8 


9 


P(X) 


I(F 


0 


1 


2 


0 


0 


0 


0 


0 


0 


3 


10^ 


0 


11 


11 


0 


0 


0 


0 


0 


0 


22 


10® 


0 


34 


34 


10 


0 


0 


0 


0 


0 


78 


10® 


0 


107 


89 


48 


1 


0 


0 


0 


0 


245 


10" 


1 


311 


229 


189 


20 


0 


0 


0 


0 


750 


10® 


2 


880 


485 


563 


124 


3 


0 


0 


0 


2057 


10® 


2 


2455 


1105 


1417 


563 


54 


1 


0 


0 


5597 


10"° 


2 


6501 


2391 


3435 


2133 


405 


13 


0 


0 


14884 


25.10® 


2 


9581 


3146 


4842 


3454 


786 


42 


0 


0 


21853 


10"" 


2 


17207 


4886 


7909 


6845 


1966 


156 


4 


0 


38975 


10"® 


2 


46080 


9949 


17087 


19132 


8196 


1146 


37 


0 


101629 


10"® 


2 


123877 


19843 


35259 


49479 


29064 


6306 


407 


2 


264239 



Table 1. The number of pseudoprimes with d distinct prime factors up to 10^®. 



For any odd m we let /(m) denote the multiplicative order of 2 modulo m, 
that is, the least power / > 1 such that 2^ = 1 mod m. If m = then 

/(m) = 1cm {/(p“*)}. Clearly fV is a pseudoprime if and only if f{N) divides 
N — 1 . Further define w{p) to be the largest exponent such that p™ | 2^“^ — 1. 

In practice it seems rare to have w{p) > I and so in the main part of the 
search we shall consider square-free pseudoprimes. We return to this point in 
section 6. 

We assume throughout that we are searching for pseudoprimes less than some 
bound X-. in our computations we took X = 10^^. 

Proposition 1. Let N he a pseudoprime less than X with exactly d prime fac- 
tors Pi < ■ ■ ■ < Pd- 

1 . For each i, f{pi) | IV — 1. 

2. Each Pi satisfies N =pi mod Pif{pi) and Pif{pi) < X. 

3. For r < d put Pr = 01=1 K- Then Pr-i-i < {X/ PrfiTd-r) p^-i-i is prime 

to f{pi) for all i < r. 

Proof. Part (1) follows immediately from the condition 2^“^ = 1 modpi. 

Since /(pi) | Pi — 1, we have N = 1 = pi mod f{pi), N = pi modpi and 
Pi prime to f{pi), so Pff{pi) \ N — pi. Further, N is not prime, so Pif{pi) < 
N — Pi < X and (2) follows. 

The Pi are in increasing order so the inequality in (3) is trivial. Since f{N) \ 
N — 1 , we have f{N) prime to N; but f{pi) \ f{N) and pi \ N, so the remainder 
of (3) follows. □ 

We consider three classes of pseudoprimes and adopt a different strategy 
for each. For pseudoprimes with a repeated prime factor we use the strategy 
of section 6 and for square-free pseudoprimes with a prime factor greater then 
X/\tfi we use the strategy of section 5. The remaining class of square-free pseu- 
doprimes is the most numerous and here we apply the main strategy, consisting 
of a precomputation and the main search, described in the next two sections. 
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d 


C? 


N 

factors 


2 




341 

11 • 31 


3 


C 


561 

3- 11 • 17 


4 




11305 

5- 7- 17- 19 


5 


c 


825265 

5 • 7 • 17 • 19 • 73 


6 




45593065 

5- 7 - 17 - 19- 37- 109 


7 




370851481 

7- 11- 13- 17- 19 -31 -37 


8 




38504389105 

5- 7 - 13- 17- 19- 37- 73- 97 


9 




7550611589521 

7- 11 • 13 • 17- 31 • 41 • 59 • 61 • 97 


10 




277960972890601 

7 • 11 • 13 • 17 • 19 • 31 • 37 • 41 • 101 • 181 


11 




32918038719446881 

7 • 11 • 13 • 17 • 19 • 31 • 37 • 41 • 47 • 73 • 631 


12 




1730865304568301265 

5 • 7 • 13 • 17 • 19 • 23 • 37 • 59 • 67 • 73 • 199 • 241 


13 




606395069520916762801 

11 • 13 • 17 • 19 • 29 • 31 • 41 • 43 • 61 • 73 • 97 • 127 • 151 


14 




59989606772480422038001 

7 • 11 • 13 • 17 • 19 • 31 • 37 • 41 • 61 • 73 • 97 • 151 • 241 • 251 


15 




6149883077429715389052001 

11 • 13 • 17 • 19 • 29 • 31 • 37 • 41 • 43 • 61 • 73 • 97 • 113 • 181 • 257 


16 




540513705778955131306570201 

11 • 13 • 17 • 19 • 29 • 31 • 37 • 41 • 43 • 71 • 73 • 109 • 113 • 127 • 151 • 163 


17 


c 


35237869211718889547310642241 
13 • 17 • 19 • 23 • 29 • 31 • 37 • 41 • 43 • 61 • 67 • 71 • 73 • 97 • 113 • 127 • 211 



Table 2. The smallest pseudoprimes with d prime factors, 2 < d < 17. C denotes a 
Carmichael number. 



3 The Precomputation 

The main strategy is considerably improved by a precomputation, which also 
proves to be of value in the sieving methods of section 7. Let A[F,Y] denote 
the set of primes g < F for which f{q) < F, and let B [X] denote the set of 
primes q for which qf{q) < X. By Proposition 1(2), the prime factors of the 
pseudoprimes up to X are all elements of B [X] . 

We needed to find the set B [lO^^] . The pairs {p, f{p)) were found by four 
overlapping methods. 
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< 10^- The list ^[l0^,oo] was obtained from the list of factors of 
numbers of the form 2^ — 1 with / < 10^ in Brillhart et al [2] (the “Cunningham 
tables” ) . 

B. 1Q3 < / < 10®. The lists A [lO^, A [lO®, 10®] and A [lO®, 10®] were 
computed as follows. For each value of /, put /' = 1cm {2, /} and let p range over 
the values = 1 mod /'. If 2^ = 1 mod p then test p for primality by trial division. 
If p is prime, it is added to the corresponding list: if not, it is a pseudoprime in 
its own right. 

C. p < 10®. The lists A [lO^, 10^] and A [lO®, 10®] were computed by letting 
p run over primes, determined as such by trial division; factorising p — 1, again 
by trial division; finding f{p) by considering the divisors e of p — 1 and testing 
whether 2® = 1 mod p; and extracting those p for which /(p) was in the desired 
range. 

D. 10® < p < 2.10^. The lists A [lO®, 10®] , A [lO®, 10®] and A [5.10®, 2.10®] 
were computed by letting p run over numbers prime to 6; computing values of 
2^ mod p for / in the desired range by successive doubling modulo p; testing 
whether / divides p — 1; and then checking p for primality or pseudoprimality 
by trial division. 



4 The Main Strategy 

For the main search we assume that fV is a pseudoprime less than the pre- 
assigned bound X and with exactly d prime factors, all distinct and less than 
X/10®: pseudoprimes not satisfying these conditions will be dealt with in sub- 
sequent sections. We obtain all such N as lists of prime factors pi, . . .,p^ by 
a back-tracking search. For suitable choices of F and Y we make use of the 
precomputed lists A[F,Y], 

We produce successive lists of pi, . . . ,pd-i recursively, looping at each search 
level over all the primes permitted by Proposition 1(3). 

At search level d — I put P = Oti Pi T = f{P) = 1cm {/i , . . . , fd-i}- 
We look for primes q such that N = Pq is a pseudoprime; that is, we require 
= 1 mod q and 2^^“^ = 1 mod P. The first condition is equivalent to 
Pq = 1 mod f{q) and the second to Pq = 1 mod L. But q = 1 mod /(q), so we 
require /(q) | P—1 and Pq = 1 mod L. We consider the possible q with /(q) = / 
in two ways, making use of a suitable precomputed A[F,Y], 

For every factor / of P — 1 which satisfies / < P, we let q run over the primes 
from A [F, Y] for which q < X/P and /(q) = /. If Pq = 1 mod L then N = Pq 
is a pseudoprime. 

For factors / of P — 1 with / > P, or for values of q greater than XjP, we let 
q run over the integers satisfying P = 1 mod / and /(q) = /. These conditions 
imply that Pq = 1 mod / and Pq = 1 mod L, so it is sufficient to run over the q 
satisfying Pq = 1 mod lcm{/, P}. If 2^ = 1 mod q and q is prime then N = Pq 
is a pseudoprime (and if q is composite then it is itself a pseudoprime). 
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We observe that small / are likely to occur often as factors of the P — 1 
and so the precomputation of the A[F,Y] gives a considerable saving. We note 
that Cipolla [3] used the factorisation of 2^ — 1 in an early computation of 
pseudoprimes. 

Testing candidates for pi for primality is required at every stage of the cal- 
culation. We found that using a table of primes up to a suitable limit produced 
a considerable saving in time. 

Applying the main search with X = 10^^ we used A [lO^, 10®] n B [lO^®] as 
the auxiliary list of primes. There were 64575 primes in this list. 



5 Pseudoprimes with Large Prime Factor 

Suppose that N = Pq is a square-free pseudoprime less than X with a prime 
factor q greater than A/10^; so we are assuming that P < 10^. We have f{q) \ P— 
1, so f{q) < 10^, that is g | 2^ — 1 for some / < 10"^. We use the lists A [lO®, oo] 
and A [lO^, 10^®] of primes dividing such numbers produced by methods A and 
B of section 3. For each q in this list, and for each P < X/q with P = 1 mod f{q), 
we test whether 2^^ = 2 mod P: if so, then N = Pq is a pseudoprime. There are 
277 pseudoprimes up to 10^® with a prime factor greater than 10®: the 9 which 
are less than 10^® are given in Table 3. 



N 


factors 


260907275113 

470968083601 

542620603069 

608041244701 

688388773637 

710663629201 

733007751851 

809041003843 

934155386445 


89 • 2931542417 
461 • 1021622741 
409 • 1326700741 
11 • 41 • 1348206751 
269 • 2559066073 
601 • 1182468601 
83 • 8831418697 
499 • 1621324657 
3 • 5 • 29 • 2147483647 



Table 3. The 9 pseudoprimes less than 10^^ with a prime factor greater than 10® 



6 Pseudoprimes with a Repeated Prime Factor 

Recall that w{p) is the largest exponent such that p™ | 2^“^ — 1. 

Proposition 2. Let p be an odd prime and put / = f{p), w = w{p). If a < w 
then f{p°‘) = f: if a> w then f{p°‘) = p°‘~'"f. 

Proof. The multiplicative group modulo p“ is cyclic of order — 1), and 

reduction modulo p maps this group onto the multiplicative group modulo p, 
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which is cyclic of order p — 1. We conclude that / | and further that the 

quotient f{p°')/f is a power of p. 

Clearly if a < w then 2^” = (2^)^ = 2^” ^ = . . . = 2 mod so /(p“) = /. 

We now claim that for a > w, 

where Xa is an integer with Xa = X^ mod p. Since X^ is prime to p by definition 
of w, Xa is prime to p. It follows immediately that the power of p dividing f{p°‘) 
is and this will complete the proof of the Proposition. 

We proceed by induction. The case a = w is immediate. Suppose now that 
a > w and 

with Xa an integer, Xa = X^ mod p. We have 

2P“+^-”/ = (l+p<^Xaf = l+p'^+^Xa + R 

where R denotes the sum of the remaining terms of the binomial expansion. 
Since p divides the binomial coefficient for l<r<p— 1, we see that the 

power of p dividing the term in R is at least for 1 < r < p— 1 

and at least p°‘^ for the final term. Now p > 3, so every term in R is divisible by 
Now a > w > 1, so 1 + 2a > a + 2, and | R; hence 

= l+p-+^Xa+l 

where Xa+i = Xa mod p. This completes the induction step and the claim is 
proved. □ 

We note that if p“ divides a pseudoprime N then f{p°‘) must divide TV — 1 
and so be prime to p. Hence we must have w{p) > a and consequently p“ is itself 
a pseudoprime. 

Suppose now that TV is a pseudoprime divisible by a repeated prime factor 
p“ with a >2. 

Lehmer [5] has shown that the only primes p < 6.10® satisfying this condition 
are p = 1093 and p = 3511, each in case with w{p) = 2. Since we require p® < 
10^®, we restrict our attention to these two values of p. (It is easy to check directly 
that these are the only two such p up to 10®'®.) We have /(1093®) = 364 and 
/(3511®) = 1755. For each value of p we consider numbers Q = 1 mod /(p®) such 
that TV = p^Q < X and for each such TV test directly whether 2^ = 2 mod TV. 

We took X = 10^®. There are 23 pseudoprimes up to 10^® with a repeated 
factor and 54 up to 10^®: those up to 10^® are given in Table 4. 
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N 


factors 


1194649 


1093'' 


12327121 


3511'^ 


3914864773 


29 • 113 • 1093'^ 


5654273717 


1093'^ • 4733 


6523978189 


43 • 127 • 1093'^ 


22178658685 


5 • 47 • 79 • 1093'^ 


26092328809 


1093'^ • 21841 


31310555641 


1093'^ • 26209 


41747009305 


5 • 29 • 241 • 1093'^ 


53053167441 


3 • 113 • 131 • 1093'^ 


58706246509 


157 • 313 • 1093'^ 


74795779241 


137 • 457 • 1093'^ 


85667085141 


3 • 11 • 41 • 53 • 1093'^ 


129816911251 


3511'^ • 10531 


237865367741 


1093'^ • 199109 


259621495381 


3511'^ • 21061 


333967711897 


1093'^ • 279553 


346157884801 


3511'^ • 28081 


467032496113 


313 • 1093'^ • 1249 


575310702877 


337 • 1093'^ • 1429 


601401837037 


1093'^ • 503413 


605767053061 


157 • 313 • 3511'^ 


962329192917 


3 • 29 • 47 • 197 • 1093'^ 



Table 4. The 23 pseudoprimes with repeated factor up to 10^^. 



7 Checking Ranges by Sieving 

We used a sieving technique to verify that the lists of pseudoprimes produced 
by the method of the preceding sections were complete in certain ranges. 

Suppose that we wish to list those pseudoprimes in a range up to X which 
are divisible only by primes in some list C of primes, all less than Y . Clearly we 
may assume that Y < X. We form a table indexed by the integers up to X and 
initially set each entry in the table to zero. For each p in £ we add logp into the 
table entries corresponding to numbers t with t = 0 mod p and t = 1 mod /(p): 
that is, t = p mod pf{p). At the end of this process we output any N for which 
the corresponding table entry is equal to logfV. Such an N has the property 
that all the prime factors p oi N are in C and that N = 1 mod /(p) for every p 
dividing N: that is, fV is a pseudoprime whose prime factors are all in C. 

From Proposition 1(2) we note that taking C = B [A] in the sieve will give 
all the pseudoprimes up to A. 

To estimate the time taken to sieve over a range we need the following result. 

Proposition 3. Fix an integer b and let f{p) denote the order of b in the mul- 
tiplicative group modulo p for b prime to p. The sum, taken over primes p not 
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dividing b, 

p pf{p) 

is convergent. 



Proof. Since the terms in the series are positive, the sum is convergent if any 
re-arrangement of it is convergent. Write 



E 



1 

pf{p) 



Et E 

/ ■' fip)=f 



1 

p' 



If b has order / modulo p, then p divides 6^ — 1 and p = 1 mod /. Let pi denote 
the k, say, distinct prime factors of 6^ — 1 which satisfy Pi > b and Pi = 1 mod /: 
the Pi will include all the primes p > b with f{p) = f. We have k < f since all 
Pi > b, and Pi > 1 + if. So 



E 

f{p) = f 

p > b 



1 



^E 



/ 



Pi 



<T.i 



•rj'* 



■log/). 



Hence 







1 + log / 
p 



and the latter sum is convergent. 



□ 



We shall use this result with b = 2. In this case the numerical value of the 
sum, computed over the primes p up to 10^, is approximately 0.31734. 

The time taken to sieve over all the numbers up to X will be bounded by 



pec 



X 

Ppp) 



<x+^E 

p 



1 

ppp) 



+ n{Y) = 0{X), 



which is an improvement over a direct search for pseudoprimes: testing the con- 
dition 2-^“^ = 1 mod N for all IV up to X would already take time O(XlogX). 

In practice, we found that the contribution of order tt{Y) from considering 
elements p of C for which there are few or no multiples of pf{p) in the range 
outweighs the contribution of order X from scanning the table and so it is 
beneficial to reduce the size of the list C as much as possible. 

We therefore consider a “large prime variation” . After sieving with C the list 
of primes up to some limit Y, we use a further technique to deal with those 
pseudoprimes which have a prime factor q greater than Y. For each prime q > Y 
in B [A], we consider all numbers P up to X/q which are = 1 mod f{q). The 
procedure now follows that of section 5. For each such P we test whether 2^'^ = 
2 mod P. If so, N = Pq is a pseudoprime. 




467 



The Pseudoprimes up to 10^® 



8 Comparison with Existing Tables 

We have checked our tables against those of Pomerance, Selfridge and Wagstaff 
[11], who obtained the 21853 pseudoprimes up to 25.10®. We extracted the 19279 
Carmichael numbers from our tables and compared them against the tables of 
[7]. In each case there was no discrepancy. 

9 Some Details of the Computations 

We ran the search procedure of the main strategy, sections 2 to 4, with upper 
limits of X = 10” for each value of n up to 13 and each value of d up to 9 in- 
dependently. As a consequence the list of pseudoprimes up to 10^® was in effect 
computed twice, that up to 10^^ three times and so on, providing additional 
checks on the computations. The computer programs were written in C and run 
on Sun 3/60 and Sparc workstations. The restriction of the search to prime fac- 
tors less than A/10^, that is, less than 10®, meant that 32-bit integer arithmetic 
could be used throughout. As a check, both on the programs and the results, 
some of the runs were duplicated using the rather strict Norcroft C compiler on 
an IBM 3084 mainframe. A total of about 2000 hours of CPU time was required. 
All the results were consistent. 

The methods of sections 5 and 6 were implemented using Pari/GP on a Sparc 
workstation. Less than an hour was required for this part of the computation. 

We used the sieving process of section 7 to check the search process up to 
10^®: this consumed about 300 hours of CPU time on an IBM 3084. The results 
were consistent with those obtained by the methods of sections 2 and 3. 

As a further check, we ran the “large prime variation” of §5 for pseudoprimes 
up to 10^® with a prime factor q in B [lO^®] with q > 10^: there are 39463 such 
primes. The lists matched those found by the search process: there were 3145 
such pseudoprimes up to 10^®. 

10 Statistics 

Let P (A) denote the number of pseudoprimes less than X, and P {d, X) denote 
the number which have exactly d prime factors. In Table 1 we give P (d, X) and 
P (A) for values of A up to 10^®. No pseudoprime in this range has more than 
9 prime factors. We have P (lO^®) = 264239. 

In Table 2 we give the smallest pseudoprime with d prime factors for d up to 
17. 

In Table 7 we give the number of pseudoprimes in each class modulo m for 
m up to 12. 

In Tables 8 and 8 we give the number of pseudoprimes divisible by primes 
p up to 97. In Table 8 we count all pseudoprimes divisible by p: in Table 8 we 
count only those for which p is the smallest prime factor. 

The largest prime factor of a pseudoprime up to 10^® is 77158673929, dividing 



9799151588983 = 127 • 77158673929 
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and the largest prime to occur as the smallest prime factor of a pseudoprime in 
this range is 3029563, dividing 

9518187116947 = 3029563 • 3141769. 

Define a{X) by P{X) = exp(log(X)“) and (3{X) by P{X) = X£{X)~^, 
where 

V log log X )' 

Pomerance [8], [9], [10] showed that a > 85/207 > 0.4106 and P > i for X 
sufficiently large: he conjectured that P tends to 1. Clearly if P is even bounded 
then a tends to 1. 

In Table 5 we tabulate the values of a and P for various values of X up to 10^^. 
We see that a is increasing over the range, but P is not obviously converging. 



X 


a(X) 


P(X) 


I(F 


0.048663 


2.466690 


10^ 


0.508262 


1.849388 


10® 


0.602306 


1.700002 


10® 


0.649319 


1.636881 


10" 


0.679908 


1.602218 


10® 


0.697435 


1.596159 


10® 


0.711006 


1.595093 


10"° 


0.721350 


1.598918 


25.10® 


0.724828 


1.601292 


10"" 


0.729621 


1.605264 


10"® 


0.736643 


1.612232 


10"® 


0.742721 


1.619440 



Table 5. The functions a and P of section 10. 



Define an odd composite integer N to be an Euler pseudoprime if 

where (-^) is the Jacobi symbol. Further define IV to be a strong, or Miller- 
Rahin pseudoprime if it passes the following test. Put N — 1 = 2“6 with b odd, 
and form the sequence 2^, 2^^, . . . , 2^ ^ = 2^“^ modulo N. The test is passed if 
either the first term is 1 mod N or there are two consecutive terms —1 mod N, 
1 mod N in the sequence. Finally define a Carmichael number to be a composite 
N for which = 1 mod N for any a prime to N. 

It is clear that if N is an Euler pseudoprime or a strong pseudoprime then it is 
also a pseudoprime in the sense we have been using. (It is also true, but not quite 
so obvious, that if IV is a strong pseudoprime then it is an Euler pseudoprime.) 
Since every Carmichael number is odd, it is again also a pseudoprime. 
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We can therefore tabulate the Euler pseudoprimes and strong pseudoprimes 
by extracting them from the tables of pseudoprimes. The Carmichael numbers 
in this range have already been tabulated in [7] . In Table 6 we give the numbers 
EP{X), SP{X) and C{X) of Euler pseudoprimes, strong pseudoprimes and 
Carmichael numbers up to X for various values of X up to 10^^. 



X 


P(X) 


EP{X) 


SP{X) 


C(X) 


I(F 


22 


12 


5 


7 


10® 


78 


36 


16 


16 


10® 


245 


114 


46 


43 


10" 


750 


375 


162 


105 


10® 


2057 


1071 


488 


255 


10® 


5597 


2939 


1282 


646 


10"° 


14884 


7706 


3291 


1547 


25.10® 


21853 


11347 


4842 


2163 


10"" 


38975 


20417 


8607 


3605 


10"® 


101629 


53332 


22412 


8241 


10"® 


264239 


124882 


58897 


19279 



Table 6. The numbers of pseudoprimes, Euler pseudoprimes, strong pseudoprimes and 
Carmichael numbers up to X. 



11 Even Pseudoprimes 

The condition 2^“^ = 1 mod N implies that N is odd. If we replace this condi- 
tion by the closely related 2^ = 2 mod N then it is possible for N to be even: 
for example, N = 161038 = 2 • 73 • 1103. Let us call such a number an even pseu- 
doprime. It is easy to see that such an N satisfies N = 2R with R odd and the 
condition becomes 2^^“^ = 1 mod R. It is then necessary that f{R) \ 2R— 1, so 
f{R) must be odd. Of the 145270 primes in B [lO^^] , 51607 have an odd value of 
/ and so are candidates for being an odd prime factor of an even pseudoprime. 

We adapted the methods of the previous sections to use this restricted set 
of possible prime factors and modified condition on N. There are only 155 even 
pseudoprimes up to 10^^: the 40 less than 10^° are listed in Table 9. We did not 
pursue this computation further. 
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m 


c 


25.10“ 


10“ 


10“ 


10“ 


5 


0 


1474 


2485 


5695 


13107 




1 


12721 


22936 


61119 


161588 




2 


2743 


4824 


12643 


32562 




3 


2685 


4768 


12198 


31381 




4 


2230 


3962 


9974 


25601 


7 


0 


2025 


3476 


8546 


20613 




1 


8730 


15868 


42605 


113703 




2 


2049 


3605 


9407 


24134 




3 


2491 


4387 


mil 


28742 




4 


2039 


3567 


9178 


23232 




5 


2258 


4030 


10315 


26717 




6 


2261 


4042 


10467 


27098 


8 


1 


12654 


22911 


60415 


158746 




3 


1295 


2180 


5646 


14522 




5 


6615 


11645 


29902 


76587 




7 


1289 


2239 


5666 


14384 


9 


1 


11395 


20644 


54852 


144736 




2 


935 


1649 


4287 


11107 




3 


318 


526 


1117 


2315 




4 


3513 


6148 


15833 


40994 




5 


937 


1634 


4197 


11025 




6 


310 


516 


1134 


2348 




7 


3505 


6209 


15987 


40745 




8 


940 


1649 


4222 


10969 


11 


0 


1690 


2930 


7610 


19271 




1 


5314 


9763 


26416 


70660 




2 


1572 


2773 


7186 


18399 




3 


1554 


2740 


7090 


18359 




4 


1603 


2739 


7084 


18273 




5 


1776 


3125 


7806 


20184 




6 


1593 


2886 


7530 


19482 




7 


1709 


3004 


7667 


19593 




8 


1774 


3114 


8049 


20740 




9 


1428 


2600 


6727 


17304 




10 


1840 


3301 


8464 


21974 


12 


1 


16281 


29360 


77269 


202532 




3 


29 


48 


90 


172 




5 


2389 


4202 


10887 


28310 




7 


2132 


3641 


9403 


23943 




9 


599 


994 


2161 


4491 




11 


423 


730 


1819 


4791 



Table 7. The number of pseudoprimes congruent to c modulo m. 
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p 


25.10“ 


10^^ 


10^^ 


lO'-" 


3 


628 


1042 


2251 


4663 


5 


1340 


2278 


5278 


12315 


7 


1763 


3044 


7586 


18452 


11 


1260 


2203 


5850 


15192 


13 


1149 


2147 


5624 


14486 


17 


654 


1152 


3100 


8557 


19 


619 


1099 


2929 


7777 


23 


272 


475 


1277 


3408 


29 


345 


628 


1638 


4414 


31 


551 


966 


2406 


6035 


37 


301 


531 


1354 


3613 


41 


237 


444 


1224 


3288 


43 


257 


446 


1081 


2750 


47 


61 


94 


235 


566 


53 


102 


181 


434 


1096 


59 


46 


75 


156 


393 


61 


162 


282 


770 


2119 


67 


103 


171 


433 


1054 


71 


119 


191 


506 


1226 


73 


135 


246 


614 


1628 


79 


76 


131 


304 


719 


83 


34 


50 


94 


190 


89 


68 


130 


282 


669 


97 


105 


179 


389 


911 



P 


25.10“ 


10^^ 


10^^ 


lO^-" 


3 


628 


1042 


2251 


4663 


5 


1474 


2485 


5695 


13107 


7 


2025 


3476 


8546 


20613 


11 


1690 


2930 


7610 


19271 


13 


2270 


3997 


9974 


24836 


17 


1756 


3018 


7708 


19572 


19 


1530 


2725 


7129 


18723 


23 


671 


1189 


3137 


8223 


29 


954 


1717 


4492 


11943 


31 


1575 


2783 


7138 


18322 


37 


1267 


2286 


5972 


15542 


41 


1269 


2238 


5931 


15579 


43 


930 


1641 


4296 


11333 


47 


254 


429 


1091 


2873 


53 


400 


707 


1878 


4797 


59 


145 


246 


631 


1704 


61 


1007 


1824 


4897 


13094 


67 


486 


830 


2156 


5793 


71 


501 


907 


2502 


6838 


73 


1104 


1990 


5069 


13296 


79 


307 


558 


1432 


3827 


83 


82 


143 


355 


867 


89 


434 


783 


2098 


5501 


97 


653 


1147 


2988 


7779 



Table 8. The number of times a prime p < 97 occurs in a pseudoprime, as any factor 
and as the least prime factor respectively. 
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N 


factors 


161038 


2 • 73 • 1103 


215326 


2 • 23 • 31 • 151 


2568226 


2 • 23 • 31 • 1801 


3020626 


2 • 7 • 359 • 601 


7866046 


2 • 23 • 271 • 631 


9115426 


2 • 31 • 233 • 631 


49699666 


2 • 311 • 79903 


143742226 


2 • 23 • 31 • 100801 


161292286 


2- 127- 199-3191 


196116194 


2 • 127 • 599 • 1289 


209665666 


2- 7 - 89- 191 • 881 


213388066 


2 • 23 • 31 • 151 ■ 991 


293974066 


2 • 73 • 631 • 3191 


336408382 


2 • 73 • 1103 • 2089 


377994926 


2 • 23 • 89 • 127 • 727 


410857426 


2 • 7 • 191 • 153649 


665387746 


2 • 23 ■ 3463 • 4177 


667363522 


2 • 7 • 5471 • 8713 


672655726 


2 • 73 • 1103 • 4177 


760569694 


2 • 1319 • 288313 


1066079026 


2 • 23 • 31 • 151 • 4951 


1105826338 


2 • 23 • 73 • 127 • 2593 


1423998226 


2 • 7 • 79 • 271 • 4751 


1451887438 


2 • 79 ■ 89 • 223 ■ 463 


1610063326 


2 • 73 • 2089 • 5279 


2001038066 


2 • 47 • 311 • 68449 


2138882626 


2 • 73 • 3191 • 4591 


2952654706 


2 • 31 • 71 • 631 • 1063 


3220041826 


2 • 73 • 103 • 233 • 919 


3434672242 


2 ■ 727 • 911 • 2593 


4338249646 


2 • 4721 • 459463 


4783964626 


2 • 7 • 23 • 73 • 271 • 751 


5269424734 


2 • 7 • 1433 • 262657 


5820708466 


2- 79- 3257- 11311 


6182224786 


2 • 23 • 31 • 151 • 28711 


6381449614 


2 • 73 • 199 • 239 ■ 919 


8356926046 


2 • 7 ■ 79 • 7555991 


8419609486 


2 • 31 • 2441 • 55633 


9548385826 


2 • 7 • 31 • 89 • 247201 


9895191538 


2 • 127 • 1289 • 30223 



Table 9. The 40 even pseudoprimes up to 10^°. 
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Abstract. Gomputing the number of Goldbach partitions 
9(n) = I n=p + q,p<q} 

of all even numbers n up to a given limit can be done by a very sim- 
ple, but space-demanding sequential procedure. This work describes a 
distributed implementation for computing the number of partitions with 
minimal space requirements. The program was distributed to numer- 
ous workstations, leading to the calculation of g{n) for all even n up 
to 5 X 10®. The resulting values are compared to those following from 
previously stated conjectures about the asymptotic behaviour of g. 



1 Introduction 

One of the most famous unsolved problems in number theory, the Goldbach 
Conjecture states that every even number can be written as the sum of two 
primes. While still being unproved that every even number n has at least one 
partition (p, q) with n = p + q, it has long been observed that the number of 
partitions grows with increasing n. Table 1 shows a few values of g. 



n 


4 6 


8 


10 


12 


14 


16 


18 


20 


22 


24 


26 


28 


30 


32 


34 


36 


38 


40 


42 


44 


46 


48 


50 


g(n) 


1 1 


1 


2 


1 


2 


2 


2 


2 


3 


3 


3 


2 


3 


2 


4 


4 


2 


3 


4 


3 


4 


5 


4 



Table 1. 



The value g{n) strongly depends on the factorization of n. As an example, 
120 = 2 • 2 • 2 • 3 • 5 yields 5(120) = 12, whereas the neighbouring even numbers 
118 = 2-59 and 122 = 2-61 give 5(118) = 6 and 5(122) = 4, respectively. 

Heuristical explanations, and formulas for g(n) based on probabilistic con- 
siderations have been derived by numerous authors in the past; a nice represen- 
tation, given by Nils Pipping in 1926 can be found in [24]: By sieving out the 
primes from the sets {3, 5, 7, 9, . . . ,n} and {n — 3, n — 5, n — 7, n — 9, . . . ,3}, one 
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can get a first approximation to g{n) by taking g(n) « n ■ P^{n), where P{n) 
denotes the probability that a number less than n is prime. But twice choosing 
a prime is not independent from each other, so a correction will be necessary: 
By first considering those p that divide n and then those being coprime to n, 
one can get two correction factors, 

P 

p-l 

p\n 




for the first case and 



TT P{P - 2) 

11 (p_l)2 



for the second. Multiplying both gives 



2 



n 

3<P<\/ 

p|n 



P- 1 




P{P - 2) 

{p-ir 



where the second product tends to the twin prime constant C 2 
Thus, it can be conjectured that 



9{n) 



2C2P^{n) 




p\n 



0.66016182. 

( 1 ) 



where the quotients of the product explain 5 (n)’s dependency on n’s factors. 
For details on the derivation of the above two factors, the reader is referred to 
Pipping’s description in [24] . 

In 1871, Sylvester [43] was the first one who described a formula close to (1). 
Since then, many authors have suggested different formulas based on (1) with 
different substitutions of the function P{n) and sometimes different correction 
factors. In 1974, Halberstam and Richer! [14] proved that 



g{n) < AC 2 




p — 1 n 
p-2 log^ n 



(^1 + 0 



/ loglogn \\ 

V logn )) 



More recently, in 1993, Deshoulliers, Granville, Narkiewicz and Pomerance 
showed that the maximal n for which equality holds in 



g{n) < 7r(n — 2) — •K(nj2 — 1) 



is 210. 

In Section 4.1, most of the formulas trying to give an exact estimation to g 
will be revisited and a statistical comparison to the computed values of g will 
be made. Section 2.1 gives an overview of past computations. In Section 3, our 
distributed implementation will be described and practical considerations and 
running times will be given. Finally, a discussion on the results and method 
follows. 
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2 Computing Values of g 

2.1 Haussner’s “Strip Machine” 

In 1896, Robert Haussner described a mechanical way to obtain values of g. 
We will shortly give a translation of Haussner’s original description [18] of his 
“partition counting machine” : 

“May I he permitted to briefly demonstrate how to eonstruct an appara- 
tus by which one can obtain all partitions of an even number < 2N with- 
out any calculation. One writes all odd numbers from 1 through 2N — 1 
equidistantly on two parallel strips, on one strip in ascending, on the 
other in descending order. The prime numbers are somehow emphasized 
on both strips. If one moves both strips lengthwise such that the number 1 
of the first strip lies opposite the number 2n—l of the second strip, where 
n < N, then all cases in which two prime numbers face each other give 
all partitions of 2n in two prime number summand; yet one only has to 
consider those prime numbers on the first strip that are < n. For greater 
convenience, after adjustment both strips should be reeled off one roll 
and wound onto another. It is easy to attach a mechanical counter that 
displays the number once unwinding has been successfully completed.” 

Algorithmically, Haussner’s strip machine can be summarized as follows: 



Algorithm 1 Haussner’s strip machine 
Input: Upper limit 2N 
1: oddpbit ^ sieve{oddpbit,2N) 

2: revpbit ^ reverse{oddpbit) 

3: for n ^ N downto 3 do 

4: g2n ^ 0 

5: for i ^ 1 to do 

6: if oddpbit[i] = 1 and revpbit[i] = 1 then 

7: g2n ^ g2n + 1 

8: end if 

9: end for 

10: output{g2n) 

11: revpbit ^ shiftleft{revpbit) 

12: end for 



Here, the strips have been substituted by two bit-arrays oddpbit and revpbit. 
After being sieved by the function sieve, oddpbit represents the odd numbers up 
to 2N such that oddpbit[i] = 1 iff 2z — 1 is prime. Its reversed counterpart revpbit 
is equal to 1 at position z iff 2(n — z) -b 1 is prime. The readjustment of the second 

= fl(2n) 



1 
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strip of Haussner’s machine is realized by the function shiftleft, which shifts the 
whole array revpbit left by one bit. 

Algorithm 1 could easily be implemented as a computer program. A few 
practical notes should be added, though. Instead of using two strips, it would 
be sufficient to only use the array oddpbit, successively checking (in line 6) if 
oddpbit[i] and oddpbit[N — z + 1] are simultaneously equal to 1. But the use of 
two bit-arrays can be advantageous, because one would in practice pack the bits 
representing the odd numbers into computer- words. After joining the relevant 
elements of the two arrays word wise by binary AND, one only needs to count 
the remaining 1-bits in order to get the value of g{2n). 

The generation of the primes below 2N will require at most O(NloglogN) 
operations and N bits of space. For each 3 < n < N, the inner loop is executed 
[rz/2] times, so Algorithm 1 computes the number of all Goldbach partitions 
of all even numbers up to 2N in 0{N'^) operations and 0{N) bits of space. It 
should be mentioned that in practice the bit-shifting “hidden” in the function 
shiftleft does affect the running time if one packs the odd numbers into words, 
because three operations are necessary to shift one word. 

Although Haussner published extensive tables including the number of Gold- 
bach partitions of all even numbers up to 5000 in 1896 [19], he never built his 
machine. Instead, he calculated his tables in a way similar^ to the following one, 
which is basically the method from which our distribution will be derived. 

2.2 The Base Method 



Algorithm 2 Base method 
Input: Upper limit 2N 
1: oddp <— genprimes{2N) 

2: for z ^ 1 to tt{N) — 1 do 
3: j 

4: while oddp[i] -|- oddp[j] < 2N do 

5: g[oddp[i] -|- oddp[j]] ^ g[oddp[i] -|- oddp[j]] -|- 1 

6: i ^ i + 1 

7: end while 

8: end for 
Output: g 



In Algorithm 2, oddp is an array containing all odd primes up to 2N. While 
the space requirements are still 0{N), Algorithm 2 only needs 0{N‘^ / log^ N) 
steps to generate the array g. However, storing the primes up to 2N directly in 
the array oddp requires approximately twice as much memory than that needed 
for the array oddpbit in Algorithm 1. So it would be appropriate to replace 

^ Haussner actually used a method more suitable for hand calculation, based on residue 
tables mod 100 (see [19] for details). 
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oddp by another array, say oddpdijf, storing (half) differences between successive 
primes instead^. This way, the memory requirements of the arrays oddpbit and 
oddpdijf would be about the same in our range. But the most space consum- 
ing part is the array g itself. In the range considered here, already three bytes 
of memory are necessary to store a single value of g. Since one must keep all 
intermediate values of g for all 2n < 2N as well as the array oddpdijf in main 
memory during the whole computation, the needed memory sums up to about 
47V-I- 2fV/ log 2N Bytes (assuming that each value will be stored in a 32-bit inte- 
ger). So, for example, a computation up to 5 • 10® would require approximately 
1 Gigabyte of main memory. At the time this work was carried out, this amount 
was only available on machines that could not be occupied for long running 
computations. 

Two years prior to Haussner, Cantor [9] had published a table with all par- 
titions of all 2n below 1000, extended to 2000 by Aubry [1] in 1896. In an un- 
published work in 1917, Weinreich [38], [45] checked Haussner’s tables, making 
use of Haussner’s idea of utilizing paper strips. In 1927, Pipping [25] published 
a corrected list of Haussner’s results up to 5000, which he had produced with 
a modular method^ suitable for hand calculation (again involving paper strips, 
see [24]). Pipping’s computations were further extended by Stein and Stein to 
200000 in 1964 and by Bohman and Frdberg [3] to 350000 in 1975 (without 
explicitly describing their ways of computation) . 

Almost exactly 100 years after Haussner’s description, Lavenier and Saouter 
[22] were the first ones to construct an impressive “version” of his partition 
counting machine, using a dedicated hardware being capable of computing 100 
values of g simultaneously. Their computations went up to 1.28 • 10® in 1998. 



3 Distributing the Base Algorithm 

In addition to its inferior running time, there is no apparent way to distribute 
Algorithm 1, so one would rather think about finding a distributed version of 
Algorithm 2. 

The major problem of Algorithm 2 is its huge memory expense, which can 
be lowered by a time/space-trade that will now be described. 

3.1 Principle 

In preparation to distributing Algorithm 2, let 2N = 2* • iTm, where Um is the 
product of the first m odd primes. The interval [1, 2N] will now be divided into 
2*“” segments sq, si, . . . , where t > r > 5 so each segment has a length 

of 2” • Um ■ Then, the primes p, q of each partition (p, q) of an even number in 
the segment Sj (0 < z < 2*“”) will origin in segments as shown in Table 2. The 
principle of the distributed version of Algorithm 2 is now as follows. 

® This requires only one byte per prime number up to approximately 3 • 10^^. 

^ Both Haussner’s and Pipping’s calculation methods don’t give an advantage if com- 
puters are available. 
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P G 


q e 


So 


Si-1 U Si 


Si 


Si-2 U Si-1 


S2 


Si-3 u Si-2 


Sj 


Si — j — 1 U Si-j 


S\i/2\ 


•Si-|i/2|-l U Si-|i/2 



Table 2. Possible subsets of partitions 



For each segment Si C [1, 2N], Sj and Si-j-i as well as Si-j will be sieved for 
primes for all j € [0, [t/2j]. For every j, all sums p+ q, p G Sj, q € Si-j-i U Si-j 
will be formed and checked for being G Si. If so, g will be incremented by 1 at 
position {p+q—i-2'~ Ilm) /2. After a segment Si has been completely processed, an 
array element g[n] will hold the number of partitions of the number i-2'~ Um + 2n. 
During this process, two things have to be kept in mind: Firstly, no additions 
must be performed when j > i — j — 1, and secondly, if j = z — j — 1 or j = z — j, 
one must additionally check that p < q. Also note that the sieving of two higher 
segments is only necessary when j = 0. In all other cases, the second highest 
segment will become the highest in the next step and so on. 

3.2 Implementation 



Algorithm 3 Processing one segment 
Input: Segment number z, z > 1 

1: {oddphijTThi) <— genprimes{si) { Get primes in highest segment } 

2: for J ^ 0 to [z/2j — 1 do { Process segments 0 through [z/2j — 1 } 

3: (oddpiojTTio) ^ genprimes{sj) { Get primes in lower segment } 

4: g ^ addhiig, i,oddpio, mo,oddphi, ttm) { Add lower/higher segment } 

5: (oddphijTfhi) ^ genprimes{si-j-i) { Get primes in 2"'* higher segment } 

6: g ^ addio{g,i,oddpio,TTio,oddphi,TThi) { Add lower/2"‘* higher segment } 

7: end for 
8: if z is odd then 

9: (oddpiojTTio) ^ genprimes{sj+\) { Get primes in middle segment } 

10: g ^ addhiig, i,oddpio, iHo,oddphi, Tihi) { Add lower/higher segment } 

11: g ^ addmidig, i,oddpio, lUo) { Add middle segment } 

12: else { z is even } 

13: g ^ addmidig, i,oddphi, i^hi) { Add middle segment } 

14: end if 
Output: g 



The function genprimes generates the primes of a segment, returning them 
along with irio/hii the number of primes found in the segment. In order to opti- 
mize the additions of two segments, three different functions addio, addmid, addu 
are called, depending on which segments are to be processed: 
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Algorithm 4 addio 

Input: Array g, segment number i, segments oddpio, oddphi, 

1: for j <— nio downto 1 do { Process primes p of oddpio } 

2: p^ oddpio[j] 

3: for k ^ nM downto 1 do { Process primes q of oddphi } 

4: q ^ oddphi[k] 

5: if p + q ^ Si then { No more q for this p } 

6: break { (for k...) } 

7: end if 

8: q[{p + q-i- 2"i7^)/2] ^ q[{p + q-i- 2’'77^)/2] + 1 

9: end for 

10: end for 
Output: q 



Algorithm 5 addhi 

Input: Array q, segment number i, segments oddpio, oddphi, T^io,i^hi 
1: for j i— 1 to TTio do { Process primes p of oddpio } 

2: p^ oddpio[j] 

3: for fc ^ 1 to nhi do { Process primes q of oddphi } 

4: q ^ oddphi[k] 

5: if p + q ^ Si then { No more q for this p } 

6: break { (for k...) } 

7: end if 

8: q[{p + q-i- 2''i7^)/2] ^ q[{p + q-i- 2’'77^)/2] + 1 

9: end for 

10: end for 
Output: q 



Algorithm 6 addmid 

Input: Array q, segment number i, segment oddpmid, TTmid 
1: for j <— 1 to Timid do { Process primes p of oddpmU } 

2: p^oddpmid[j] 

3: for k <— j to Timid do { Process primes q of oddpmU } 

4: q^oddpmid\k] 

5: if p + q £ Si then { If so, increment q } 

6: q[{p + q-i- 2"i7^)/2] ^ q[{p + q-i- 2"i7^)/2] + 1 

7: end if 

8: end for 

9: end for 
Output: q 



3.3 Practical Considerations and Rnnning Times 

The reason for choosing the length of the segments to be 2^ Um is that one can 
efficiently apply a segmented sieve as given in [5] or [2] in order to generate the 
primes in the segments. We choosed r > 5, so the segment lengths are divisible 
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by 32, which was the base word length. The actual implementation of the sieve 
is described in [30] . 

The running time of Algorithm 3 is essentially determined by the addition 
operations. Each call to one of the adding functions will cause at most c-l^ j log^ I 
operations, where I = abbreviates the segment length and c is a constant. 

So for one segment Si this will sum up to 0{i ■ P /\o^ 1) operations. Therefore, 
the processing of all segments (starting with S 2 ) will take about 






{2Ny 



i=2 log ^ IS log 

By approximating the sum with 

2N . , 

dx 1 



2 2N 



= 






1 2 



log 



2N 



2N 

I 



1 



1 



log I log N 



we finally get that the number of all partitions can be determined in 0(fV^/ log 1) 
operations. This suggests to take the segment length as large as possible (which 
was expectable). 

The space requirements of Algorithm 3 are determined by the array g, which 
now takes 0{l) bits of space. An additional 0{l) bits is needed for the two prime 
arrays oddp and the space for the sieves, but this is in practice negligible. 

In the actual implementation, we took m = 5, r = 6 and 2N = 500660160, 
so I = 2® • 3 • 5 • 7 • 11 • 13 = 960960, giving a total space of around 2MB. The 
program was distributed to 7 Sun Ultral, 6 Sun4 workstations and 3 PC’s. The 
total running time was approximately 70 days. Due to the quadratic running 
time, processing the last segments already took about 3 days. 



4 Results 

The resulting values of g have been checked against those of [22] up to 1.28 • 10®, 
without finding any discrepancies. 

There is a relatively easy possibility to check the resulting values by using 
an “adapted version” of Landau’s [21] summatory function 

X 

H{x) = ^ G{n) , 

n—1 

where G{n) = #{(p, g) ] n = p + q} (so for even n one has g{n) = ]’G(n)/2]). 
Landau also defined H for odd numbers, giving G(2k+ 1) = 1, if 2fc — 1 is prime 
and 0 otherwise. 

In our case, we define 



Hx) = 9{n) . 

6<n<a: 
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Since 



h{x) 



- E E 1 

= E - p) - 7 t(p) + 1) 

3<P<f 



= E 

3<P<§ 

= E ^i^-p) 

3<P<§ 

= E ^(^-p) 

3<P<f 



E E 1 

3<P<f 3<p<f 



4f)(4|) + l) 
2 



1 + 7t(|)-1 



2 



one can check the computation by calculating the sum on the last line, subtract 
the quotient and compare the result to the computed h{x).^ 

For X = 500660160 it turned out that J2s<p<^ tt{x—p) = 277532324737949, 
{tt^{x/2)-tt{x/2))/2 = 93795323525751 and therefore h{x) = 183737001212198 
which was identical to the directly computed X)6<n<a;5(’^)- checking pro- 
gram for the summation of the tt(x — p) was based on the same sieve program 
as used above, appropriately modified. 

Figures 1 and 2 show plots of g{n) for even n up to 160160 and between 
500500000 and 500660160, where the stronger lines visible correspond to numbers 
divisible by smaller primes (as a consequence of the quotients of (1)). 

The maximal value of g{n) in the range considered was 3977551, taken at 
n = 497668710 = 2 • 3 • 5 • 7 • 11 • 17 • 19 • 23 • 29. 

In [41], it was conjectured that g : 2N ^ N is surjective. Up to 500660160, 
the smallest n that didn’t occur as a value of g was 2166940. 



4.1 Comparisons to Previously Stated Conjectures on g 

Sylvester in 1871 was the first one to formulate a conjecture about the asymptotic 
behaviour of g. Here, we will shortly revisit his formula along with the ones that 
have been suggested since then and later compare them to the computed values 
of g. In the following, gx will denote a suggested formula meaning gx{n) ~ g{n), 
where the index X will abbreviate the author’s name. 



Only given in a short abstract in [43], Sylvester describes® 



9Sv{n) 





p-2 ■ 



® Actually this is only a 1-error-detection. 

® It is not quite clear whether Sylvester exactly meant gsy The interpretation of the 
abstract [43] is due to Shah/ Wilson [35]. 
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Fig. 1. y(n), 6 < n < 160160 
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Fig. 2. 5 (n), 500500000 <n< 500660160 
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Without knowing of Sylvester’s work, Stackel suggested in 1896 [36] two different 
(asymptotically equal) formulas 



9Sti{n) 



7T^(n) 

(p{n) 



and 



9St2{n) 



(7r(n 



-Vn)- 7r(v^))^ 
n — 2y/n 



n 

ip{n) 



where (p is Euler’s function. Without saying that Stackel’s formulas are at all 
right, Landau [21] shows indirectly in 1900 that a correction factor 7r^/105C(3) 
must in any case be multiplied, leading to 

9La(n) = 0.772...-^^ . 

ip{n) 



In 1915, Brun [6] gave 



gBr{n) = 1.5985- 

log n 




p-2 ■ 



One year later, Stackel [37] corrects this to 



»(..)= 2. 



7T^(n) 

n 



P- 1 

p-2 ’ 





where the first product is identical to the second factor C 2 of (1). Unfortunately, 
Stackel is mostly cited for his attempt 9sti, although he was apparently the 
first one to give the most likely asymptotically correct formula gst- Before 1919, 
Hardy and Littlewood (published in 1919 [15]), without knowing Stackel’s work 
of 1916^, gave a formula asymptotically equal to Stackel’s: 



gHL{n) = 2C2- — 2 — 
log n 




p\n 



After being requested by Hardy and Littlewood to check their formula, Shah 
and Wilson [35] replace log^ n by log^ n — 21ogn: 



9swi{n) = 2 C 2 - — 2 — 
log n 



n 

— 2 log n 




P- 1 

p-2 ’ 



^ Hardy and Littlewood mention in 1922 [17] that they “have until very recently been 
unable to consult' it. 
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after first suggesting 



gsw2{n) = 2C2 - — 2 ^ H 

log n-21ogn + 2- - 



P- 1 
P-2 



Finally in 1942, Selmer [33] gave an average approximation 

dx T-r P — 1 



9 Sei{n) = 2C2 



'o log(f + a;)log(f -a;) 



n 



p\n 



■ P-2 ’ 



which he found by taking the derivate of Landau’s summatory function, re- 
placing n/logn by li{n). Selmer also considered the second term of Riemanns 
approximation to 7r(a;), leading to gse 2 {n) = 



C 2 



1 



/o log(f -kx)log(f -a;) 



2 - 



1 



1 






dx n 

p — 2 



In the following comparison, we will also consider exchanging 7r^(n) in Stackel’s 
formula by the square of Riemann’s 









Values of R can be easily computed by taking Gram’s formula 



R{x) = 1 + ^ 



log'^s 



fc=i 



k\kC{k+l) ’ 



and using precomputed values of C(^)- In our range, we took 70 iterations of the 
sum. So we get “Riemann’s approximation” to g: 



gR(n){n) = 2 C 2 



R^{n) 

n 




P- I 

P-2 ■ 



(which we will denote by 5 _r(„) in order not to accidently imply that Riemann 
himself ever gave it) . 

In Tables 3, 4 and 5, we list the real errors of the corresponding summatory 
functions hx{2N) along with their relative errors and the average, absolute, 
relative errors 



1 

N -2 



E 

6<n<2AT 



\gx{n) - g{n)\ 

g{n) 



where 2N denotes our upper limit 500660160. Due to their high computational 
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Table 3 . (h{2N) = 183737001212198) 



X 


hx(2N) - h{2N) 


hxi.^ly)-h{2N) 


iv^E 


\9x (ri)-ff(n)] 


h{2N) 




Sy 


65622453672945 


35.70 




35.28 


HL 


-19119410676636 


-10.41 




10.69 


SW2 


-482589183065 


-0.26 




0.28 


R{n) 


375967802411 


0.20 




0.22 


SWl 


-291436375645 


-0.16 




0.17 



Table 4. (/i(10®) = 1671879782) 



X 


hx{llf) - 7i(10®) 


hx(10“)-h(10“) 


1 Isx (")-9(")l 


wTo^l 


499998 5 (n) 


Sy 


473784719 


28.34 


27.51 


HL 


-255394009 


-15.28 


15.83 


SW2 


-7400835 


-0.44 


1.03 


R{n) 


7885603 


0.47 


1.00 


St 


7936767 


0.47 


0.99 


Sel 


5435703 


0.33 


0.99 


SWl 


-3419480 


-0.20 


0.93 


Se2 


352560 


0.02 


0.89 



Table 5 . {Ah = 457957086522) 



X 


Ah — Ahx 


{Ahx - Ah)!Ah 


1 \9x (Ti)-g(n)] 

330081 Q(n) 


Sy 


165533832724 


36.15 


36.15 


HL 


-46352189108 


-10.12 


10.12 


SW2 


-1145470091 


-0.250 


0.250 


St 


881922554 


0.193 


0.193 


R(n) 


869907910 


0.190 


0.190 


SWl 


-696379629 


-0.152 


0.151 


Sel 


29737472 


0.006 


0.041 


Se2 


-13166618 


-0.003 


0.040 



costs, we only compared Selmer’s and Stackel’s approximations gsei,gse 2 and 
gst in the ranges [6, 10®] and [5 • 10®, 27V]. The corresponding values of gsei{n) 
have been determined by numerical integration of 



pn—2 



du 

log u log(n — u) 



instead of its asymptotical equivalent given above (see also [33], page 6). 
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The computation was done by using Mathematica’s NIntegrate- routine, 
double checked by a second program. Tables 4 and 5 list the resulting values as 
given in Table 3, but restricted to the above ranges, this time including Selmer’s 
and Stackers (using exact tt— values) approximations. Except for the real errors, 
all values are in percentages. 

In Table 5, Ahx abbreviates the terms hx{‘2N) — hx{5 ■ 10® — 2). 



4.2 Discussion 

The relatively bad approximation of g by taking 1/log^ n as P{n) as in Hardy/- 
Littlewood’s formula seems to be due to that although asymptotically being 
equal, 7r(n) is not very accurately described by n/logn. But neither Stackel’s 
formula, using exact values of 7r(n), nor the approximation gn(n) give as accurate 
values as Selmer’s do. His second suggestion gse 2 yields very good results. It could 
be worth to also consider the next term of Riemann’s 7r-formula in Selmer’s 
integral, though this does again increase the already high computing costs of 
his estimation. Under the “easy computable” formulas, it was a bit surprising 
that Shah and Wilson’s gswi gave values superior to those given by 5 _r(„) in all 
ranges. 

As for the method used, a decrease of the space needed by Algorithm 2 
was essential in order to be able at all to compute g{n) up to our limit of 
5T0®. Depending on the development of the memory/processor speed ratio in the 
future, this might well change. But it seems likely that using a greater number 
of small machines will still be cheaper for a longer time than incorporating 
supercomputers with very limited time access. 



Remark 

As kindly communicated to the author, Y. Saouter [32] has recently announced 
another way to compute values of the function g. Though the algorithm does ex- 
tend the memory requirements of Algorithm 2, it promises a substantial decrease 
of computing time. 
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1 Introduction 

The construction of group ring elements that annihilate the ideal class groups 
of totally complex abelian extensions of Q is classical and goes back to work of 
Kummer and Stickelberger. A generalization to totally complex abelian exten- 
sions of totally real number fields was formulated by Brumer. Brumer’s formu- 
lation fits into a more general framework known as the Brumer-Stark conjec- 
ture. We will verify this conjecture for a large number of examples belonging 
to an extended class of situations where the general status of the conjecture is 
still unknown. We assume throughout that is a totally real basefield and K 
is a totally complex extension field, abelian over k. Let wk denote the num- 
ber of roots of unity in A', m = [fe : Q], and G — Gal{K/k). We also let 
S — S{K/k) — . . . , Pi; • • • ! Pt}; where denotes the archimedean 

prime corresponding to the ith embedding of k into R, and pi, . . . are pre- 
cisely the prime ideals in k that ramify in K. For each a ^ G, we define a 
corresponding partial zeta-function 

( 1 ) 

ao.=o' 

where the sum is over all integral ideals a of k relatively prime to the ramified 
primes pi, . . .,pi and having the same Artin symbol {K/k, a) — Ua — a. The 
infinite sum on the right side of (1) converges only for > 1, but cr) 
has a meromorphic continuation to all of C with exactly one (simple) pole at 
s = 1. In particular, cr) is analytic at s = 0, and based upon work of 
Klingen [K] and Siegel [S], we know that C5(0,cr) g Q. A more refined result, 
due independently to Deligne and Ribet [DR], Barsky [B], and Cassou-Nogues 
[CN], states that G 21 for every cr £ G. Based upon this, the group 

ring element 

7 = iK/k -wk'^ 

creG 

lies in Z[Gj. Following Hayes [HI], we refer to j as the Brumer element of the 
extension K/k. The “anti-units” of K, denoted by A'®, are the elements a £ A'^ 
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having absolute value one at all archimedean primes of K. Let 05 be an arbitrary 
fractional ideal in K. We may now state the 

Brumer-Stark Conjecture: There exists an anti-unit a E /b® sueh that 
(a) = 05'’' and is abelian over k. 

Brumer originally conjectured that 7 annihilates the ideal class group of 
K (i.e. that 05'’' is always principal). The additional feature that an anti-unit 
generator of the principal ideal 05'’' can be found whose Wif th root generates an 
abelian extension over k is due to Stark. 

Before describing our computations, we hrst give a brief summary of the 
present state of the conjecture. The Brumer-Stark conjecture has already been 
proved in the following cases. 

(i) If = (Q , using Stickelberger’s Theorem (see [T2], p. 109). 

(ii) If [A' : fe] = 2 [Tl]. 

(iii) If G = Z 2 X Z 2 in general, and when G is of exponent 2 and has order 
2^ > 4, assuming Kfk is a tame extension [Sal]. 

(iv) If the class number of K is one, since the conjecture is always true for 
principal ideals [Tlj. 

(v) If |G| = 4 and K/k is a sub-extension of a non-abelian Galois extension 
A'/feo of degree 8 [Tlj. 

(vi) If K/k is a sub-extension of an abelian Galois extension K/ko, and the 
Brumer-Stark conjecture is already known to be true for K/ko ([Sa2],[H2]). 

Wiles made very important progress towards proving Brumer’s part of the con- 
jecture in [Wj. For each prime p, he formulated a sub-conjecture for the p— part 
of the ideal class group of K, and showed that Brumer ’s conjecture follows if 
the sub-conjecture can be proven for every prime p. Following Wiles, Greither 
[G] has identihed a large class of “nice” extensions and has proved that in these 
extensions the Brumer element annihilates the p— part of the ideal class group 
of K for all odd primes p. Working under these same restrictions, Popescu [P] 
has used Greither’s results to deduce Stark’s part of the conjecture as well. The 
prime 2 presents special dilhculties because all of these results rely upon the 
Main Conjecture of Iwasawa Theory in a crucial way. Based upon this summary, 
we can describe the hrst general class of situations still unproven. The smallest 
baseheld k would be real quadratic by (i). Since 2 always divides the relative 
degree [A' : k], the smallest unproven case would be where G = Z 4 by (ii) and 
(iii). Therefore [A' : Q] = 8 , and we restrict ourselves to those helds K whose 
class number exceeds one (by (iv)) and where K is non-Galois over Q (by (v) 
and (i) and (vi) combined) . The suggestion that this particular class of situations 
be studied numerically was already made by Tate in 1981 ([Tl], p. 15), but a 
serious computational study has only become feasible in recent years with the 
availability of packages such as PARI/GP [BBBCO] and KANT [DFKPRSW]. 

We present our computations according to the following plan. In section 2, we 
describe a simple method that produces an abundant supply of totally complex 
Z 4 extensions over any totally real baseheld. Section 3 contains our algorithm for 
computing the Brumer element 7 , which is uniformly applicable over any totally 
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real basefield. Section 4 gives a description of the computations required to verify 
the Brumer-Stark conjecture. Finally, a detailed example is presented in section 
5, and section 6 contains tables and comments summarizing our computations. 

2 Generating Z 4 Extensions 

The following theorem appears in a paper of Nagell [N]. Let k be an arbitrary 
basefield. Nagell proves (Thm. 3, p. 351) that any cyclic Z4 extension over k can 

be generated by a root /? of the form 6(1 + + VT+ c^) where b,c ^ k. For 

our purposes, we assume is a totally real number field and we can ensure that 
K — k{j3) is totally complex by choosing b — —1. The quartic polynomial that 

(1 + c2 + ^/T + c2) satisfies is 

f{x) - + 2{1 + c^)x^ + c^{l + c^). (2) 

Let c ^ Ok, and assume that 1 + c^ ^ k"^. Then f{x) will be irreducible over k[x] 
by Theorem 2 of [KW] and any root of f{x) will generate a totally complex Z4 
extension K over k (see Thm. 3(h) of [KW]). 

3 Computation of the Brumer Element 

With a relative extension K/k defined by an irreducible polynomial f{x) as in 
section 2, one can use the tools of a computer package (we used PARI/GP) to 
compute the number of roots of unity wk, and the relative discriminant V{K/k). 
The primes pi, . . . , dividing V{K/k) are exactly the finite primes appearing in 
the set S. The only real task remaining in the computation of the Brumer element 
is the calculation of C5(0, Given a ^ G, we need a nice characterization of all 
integral ideals a relatively prime to V{K/k) which have Artin symbol (Ja — cr.K 
beautiful characterization is provided by the Artin Reciprocity Law [Ha] which 
is most elegantly formulated in terms of the conductor \{K/k) of the extension 
K/k. The conductor of a totally complex abelian extension of a totally real 
number field has the form 



f(A'A) = fpW...p(^), 

where f is an integral ideal in k which has the exact same prime divisors as 
V{K/k). With respect to the modulus f{K/k), we obtain a partition of all frac- 
tional ideals in k relatively prime to f into a finite number of classes. These 
classes form an abelian group, called the ray class group mod f{K/k), and de- 
noted by G{f{K /k)). In general, several classes will correspond to a single au- 
tomorphism (T ^ G via the Artin map, and C5(®; is formed by summing over 
exactly the integral ideals in these classes. Even with this problem solved, we still 
need to analytically continue C5(®; in order to compute it at s = 0. The best 
known method to date is to decompose C5(®; into a finite sum of “sector zeta- 
functions” and find an analytic continuation of these latter functions. Shintani 
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[Sli] accomplished this over any totally real baselield and found an explicit eval- 
uation of the sector zeta-functions at s = 0 in terms of Bernoulli polynomials. 
The resulting formulas, as they stand, are impractical from an algorithmic point 
of view. On the other hand, one can use Shintani’s evaluations in conjunction 
with a geometric method involving “convexity polygons” to obtain an eiiicient 
algorithm over a real quadratic baseiield [HI]. This method can be generalized 
to any totally real baseiield k of degree m over Q by taking the convex closure of 
a set of lattice points in R™. Because of the resulting geometric complications, 
this method already has serious problems from an algorithmic standpoint when 
m = 3 (see [Kh], p. 276). 

We use an alternate method which relies upon the decomposition of (s{s, o") 
into a sum of L-functions. The analytic continuation of the latter type of func- 
tion is classical and dates back to Hecke. Recently, a very eiiicient method to 
compute L-function values has been used to test a related conjecture of Stark 
([DST],[Ro]). We use this method here, which is based upon a formula due inde- 
pendently to Lavrik [L] and Friedman [Fj. The relevant L-functions are defined 
from characters y : G{f{K/k)) — on the ray class group mod f{K/k). A 
given character will have a conductor of the form f(y) = f^fx.oo, where is an 
integral ideal dividing f, and fx,oo is a formal product of archimedean primes 
taken from the set We always work with the primitive version 

of X (still denoted by y) , which is defined on the ray class group mod f (y) . The 
corresponding L-function is defined by 

Ms, a) = n Ms) > I, 

with the product taken over all prime ideals in k relatively prime to f^. If we 
multiply L{s,x) by the Euler factors corresponding to primes that divide f but 
not fx (there are potentially no such primes), we obtain a related function de- 
noted by Ls{s, y). Class field theory gives a correspondence (discussed in greater 
detail below) between a particular subset Xif of characters on G{f{K/k)) and 
the abelian extension K/k. In fact, the characters in Xif form a group that is 
isomorphic to Gal(/F / k) . The decomposition of (s {s, M into a sum of L-functions 
mentioned above is 

Cs {s,cr} = x{a}Ls{s,x), (3) 

i i xexK 

where a is any ideal relatively prime to f such that Ua — cr. All of the L-functions 
defined above have meromorphic continuations to the whole complex plane and 
all are analytic in particular at s = 0. Let r(y) denote the order of the zero of 
the function L{s, y) at s = 0. 

Proposition 1. Assume [fe : Q] = m > 2. For a given eharaeter y defined on 
G{f{K/kj), we have r(y) = 0 if and only if fx,oo — 

Proof. We refer to [DT] for the basic facts used here. For the trivial character, 

oo = 1 and r(y^) = m — 1 which is greater than 0 by assumption. If y is 
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non-trivial, then r{x) — m — q, where q is the number of archimedean primes in 
the formal product fx, co- 
if L{0,x) — 0 for a given Xy then clearly Ls{0,x) = 0 as well. We will have 
L{0,x) — 0 for at least half of the characters in Xif and so the following restric- 
tion is important. Let ^k,co denote the subset of characters y £ Xif such that 
fx,co — Then equation (3) specializes to 

^ x(a)T5(0,x). (4) 

^ xexK.oc. 



In the following discussion, we focus on a fixed character y £ ^k,co ■ Before we 
can give the Lavrik-Friedman formula for the non-zero complex number L(0, y), 
we need a few preliminary definitions. Let Ax — ydfeNf^^/ 7 r™, where dk is the 
discriminant of the field k and N fx is the norm of the integral ideal fx - Let a„ (y) 
denote the finite sum x(tt) taken over all integral ideals a of norm n that are 
relatively prime to fx- Finally, let 






S-\-ioo 



z+l 



dz 



2 J J z — s 

for any J > 1. Then (see equations (4) and (5) of [DT]) 

1 



i(0,x) = 



n>l 



an{x)f{ +W{x)anix) 



( 5 ) 



where W{x) i® complex number of absolute value one known as the “Artin 
root number” of y. Let denote the set of primes dividing f but not fx - If there 
exists a prime p £ such that y(p) = 1, then L 5 ( 0 ,y) = 0. Otherwise 



L 5 ( 0 ,y) = L( 0 ,y) [J (1 - y(p)) ^ 0 . 

peiy 



We would like to point out that Louboutin [Lo] has arrived at an equivalent form 
of equation (5) independently and has used it to compute relative class numbers 
of CM-fields. He actually gives a formula for L(l,y), but the two values are 
related by L(0,y) = 7 r“”‘'^^IT(y)Ap^L(l,y) via the functional equation. Using 
equation (5), we can approximate L(0,y) to any desired degree of accuracy 
by taking enough terms. We refer to [Lo] for detailed error bounds. The root 
number IU(y) is a finite sum with <i>{fx) terms. We refer to [DT] and [Lo] for its 
computation. 

The method we have described thus far to compute C5(0; <^) i® applicable to 
any totally complex abelian extension K over a totally real field k. Throughout 
the remainder of this section we will assume that G = E 4 . Since G = E 4 , 
there is a ray class group character y of order 4 corresponding to K/k, whose 
conductor f(y) is equal to f{K/k). The conductor of the trivial character Xo i® 
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of course 1. The conductors of the two characters — X a^nd = X^ are equal 
since they are conjugate to each other. The conductor of the character X 2 — X^ 
contains no archimedean primes and is equal to the relative discriminant of 
the relative quadratic extension k' — k{x'l + c^)/k. The conductor-discriminant 
formula [Ha] gives us the relation V{K/k) — V{k^ /k)f^ and thus an immediate 
determination of f. We can now compute G{f{K /k)), but we still have to identify 
the exact set of characters Xif = {Xo ? Xi ? X 2 ; X 3 } corresponding to the extension 
K/k. To do this, we need to generate a subgroup of index 4 using relative norms. 
Based upon the following result of Bach and Sorenson [BS] (which assumes the 
ERH), we don’t have to work too hard. Let 

C= (41og|dK| + 2.5[/F :Q] + 5)2 

and let T denote the set of prime ideals in k of degree 1 over Q not dividing f 
and having norm < C. Let H be the subgroup in G{f{K/k)) generated by the 
ideals where p runs through T and fp denotes the residue degree of p in 
K/k. Then [G{f{K /k)) : H] — A and the four characters on G{f{K/k)) which are 
trivial on H make up the set Kk- We have L{s, Xi) — Ls{s, Xi) for ^ = 1, 3 and 
Ls{0, Xi) — 0 for i — 0, 2. Since L(0, xj — L{0, x^), we hnally obtain 

wkCs{0, <t) = ^(5f(x7(a)L(0, yj)) (6) 

from equation (4). Recalling that G 21, we just need to compute 

L( 0 ,Xi) fo high enough accuracy to determine the integer on the right side of 
(6). 

Even though we haven’t made a detailed comparison between our method 
and the method in [HI], we believe that our method performs equally well when 
m — 2 and certainly much better when m > 2 . 

4 The Numerical Verification of the Conjecture 

We begin with 

Proposition 2. Let Q5i, . . . , be a system ofZ,[G\-generators of the ideal elass 
group of K . Then the Brumer-Stark eonjeeture is true for every fraetional ideal 

05 in K if and only if it is true for eaeh ideal 05i with 1 < i < s. 

Proof This is a direct consequence of the properties of the subgroup of fractional 
ideals verifying the Brumer-Stark conjecture (see [Tl], p. 7). 

Thus, it is enough to verify the conjecture for a hnite set of ideals 05i, 1 < t < s. 
Eurthermore, a system of Z[G]-generators can easily be extracted from a system 
of ideals that generate the ideal class group over Z . 

To prove that a given fractional ideal 05 in K verihes the Brumer-Stark 
conjecture we proceed as follows. We hrst compute the ideal 05'’' and its class in 
the class group. If it is not principal, then the conjecture is false. Otherwise, we 
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compute a generator /? of 05'’'. In general, this generator is not an anti-unit and 
requires modification. If an anti-unit generator a exists, then we have a — e/3 for 
some unit e of K. The unit e is determined using the process described below. 
We assume that G = Z 4 = < cr > and [k : Q ] = 2 throughout the remainder of 
this section. 

Let K C C, i — 1, 2, 3, 4 be four non complex conjugate embeddings. 

For a E K, let \a\i — be the normalized absolute value. Consider the 

classical logarithmic embedding 

A : 

a 1 -^ (log|a|i,log|a| 2 ,log|a| 3 ) . 

The anti-units are contained in the kernel of A, and if a exists then 

A(e) -|- A(/?) = 0. 

Let 1 1 • 1 1 be the Euclidean norm on R® and let b be the minimal non-zero norm 
||A(w)|| where u ranges through the units of K. Then the unit e, if it exists, is 
the unique unit (up to some root of unity in K) satisfying 

||A(e) + A(/?)||<6/2. 

This unit can be found using computation with real numbers, however once it 
is found, we still need to check that a possesses the required properties. The 
following proposition allows us to verify that a is an anti-unit. 

Proposition 3. a £ A'® if and only = 1. 

Proof. The automorphism is the unique complex conjugation of the extension 
K/k, thus \a\i — \a'^ |i for all i’s, so |i = \a\f and also is a positive 

real number. Now assume that a is an anti-unit. Then \a\{ — 1 for all ris, and 
thus — 1. On the other hand, if — 1, then \a\f — 1 for all ris and a 

is an anti-unit. 

Since e is unique up to a root of unity in A', so is a. Therefore, the condition 
that generates an abelian extension over k does not depend upon the 

choice of a. The next proposition allows us to verify this condition. We first note 
that wx — 2 for all of the fields suggested by Tate for study (see Introduction) . 
To see this, let A be the field generated over Q by the roots of unity contained 
in A'. If wx > 2, then Lk is a totally complex sub-extension of K/k. Therefore 
Lk — K and K/Q is abelian, which gives a contradiction. 

Proposition 4. Kf\/a) is abelian over k if and only if a'^~^ E K^ . 

Proof. This follows directly from Prop. 1.2, p. 83 of [T2]. 

Note that all of the required computations are done with exact objects and 
therefore give a complete verification of the Brumer-Stark conjecture for all 
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of the examples tested and not just a verification up to the precision of the 
computation! 

Since the prime 2 seems to play a special role in the conjecture, we make 
the following dehnition. We call the maximum power of 2 that can be factored 
out of the Brumer element j the “2-part of 7”. We have actually tested the 
conjecture in such a way as to see how much of the 2-part of 7 is really needed. 
More precisely, let 2® be the 2-part of 7. We have searched for the smallest 
non-negative integer i such that the conjecture is true with 7 replaced by 2®“®7. 
These results are described in the last section. 

5 An Example 

Let k — Q(v^) and let K be the held generated by the polynomial (2) with 
c = 3 -|- 3^2. The discriminant dx is 2®^ • 17® and the conductor \{K/k) is 
where p2 = (V2) is the unique prime ideal above 2 and pir = 
(1 -|- 3V2) is one of the two prime ideals above 17. The held K is generated over 
(Q by an algebraic integer 9 satisfying 

-h 406»® -h 3806'"^ -f 13606»® -f 1666 = 0. 

The held K is not Galois over Q and its class group is isomorphic to Z20 x Z2. 
Moreover, its class group is generated over Z[G] by the class of the ideal 

Q5i = 3Giv + (d + l)Gif. 

The Galois group G of the extension K /k is generated by the automorphism 

(7 : 4- (56»^ -h 2356»® -f 29786»® -f 89356») . 

567 

We compute the Brumer element and hud that 

7 = 8— 16(7 — 8(7® -|- 16(7® = 2® (1 — 2(7 — (7® -|- 2(7®) . 

We start by testing 7/8, but the ideal 05))^® is not principal. Next, we look at the 
ideal 05))^^ which is principal, and using the method described in the previous 
section we hud that it is generated by the anti-unit 

a = (1106»^ -h 986»® -f 40366»® -f 37246»"^ -f 283466»® -f 292886»®-h 

5103 

530566» -h 63679) . 

However, the algebraic number a'^~^ is not a square in K, so the condition 
in Proposition 4 is not satished. Finally, it is clear that all the conditions are 
satished for 7/2. 

Theorem 1. The Brumer-Stark conjecture is true for this extension, it is even 
true if one replaces the Brumer element 7 by 7/2. 
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6 Tables and Summary 

We used the quartic polynomial (2) with k ranging through the real quadratic 
fields of discriminant < 500 and c ranging through the algebraic integers in k of 
T 2 -iiorm < 200 with 1 + ^ k"^ . Discarding the fields K obtained in this way 

that have class number one, are Galois over Q , or have discriminant > 10^®, 
and keeping only non-isomorphic fields, we end up with a list of 379 fields. The 
Brumer-Stark conjecture has been tested for each of these field extensions using 
the package PARI/GP [BBBCO]. 

Theorem 2. The Brumer-Stark eonjeeture is true for all 379 field extensions 
listed in the tables below, 

in the following tables, we list the discriminants d of the real quadratic fields 
considered and the corresponding elements c. We set tc = (1 + Vd)/2 if d = 1 (mod 4) 
and IX = Vd/2 if d = 0 (mod 4). The ring of integers of the real quadratic field k of 
discriminant d is Ok = Z + Ztc. 



d 


Values of c 


5 


A-\- Lo ^ 1 -|- 2ct^, 2 -|- 2ct^, 3 -|- 2ct^, 4 -|- 2ct^, 5 -|- 2ct^, 

6-|-2ct^, 8-|-2ct^, — l-|-3ct^, 3ct^, l-|-3ct^, 2-|-3ct^, 3-|-3ct^, 4-|-3ct^, 5-|-3ct^, 6-|-3ct^, — l-|-4ct^, 
4a;, 1 + 4a;, 2 + 4a;, 3 + 4a;, 4 + 4a;, 5 + 4a;, 6 + 4a;, —2 + 5a;, —1 + 5a;, 5a;, 1 + 5a;, 
2 -|- 5a;, 3 -|- 5a;, 4 -|- 5a;, 5 -|- 5a;, — 2 -|- 6a;, — 1 -|- 6a;, 6a;, 1 -|- 6a;, 2 -|- 6a;, 3 -|- 6a;, 
4 “h 6a;, To;, 1-1-Ta;, 2-hTa;, — 24- 8a;, 8a; 


8 


2 -|- a;, 3 -|- a;, 4 -|- a;, 5 -|- a;, 6 -|- a;, T -|- a;, 8 -|- a;, 1 -|- 2a;, 2 -|- 2a;, 4 -|- 2a;, 5 -|- 2a;, 

6 -|- 2a;, T -|- 2a;, 8 -|- 2a;, 9 -|- 2a;, 1 -|- 3a;, 2 -|- 3a;, 3 -|- 3a;, 4 -|- 3a;, 5 -|- 3a;, 6 -|- 3a;, 

7 + 3a;, 8 + 3a;, 9 + 3a;, 1 + 4a;, 2 + 4a;, 3 + 4a;, 4 + 4a;, 5 + 4a;, 6 + 4a;, 7 + 4a;, 

8 -|- 4a;, 1 -|- 5a;, 2 -|- 5a;, 3 -|- 5a;, 4 -|- 5a;, 5 -|- 5a;, 6 -|- 5a;, 1 -|- 6a;, 2 -|- 6a;, 3 -|- 6a;, 

4 + 6a;, 5 + 6a; 


12 


3 -|- a;, 4 -|- a;, 5 -|- a;, 6 -|- a;, 7 -|- a;, 8 -|- a;, 9 -|- a;, 1 -|- 2a;, 2 -|- 2a;, 4 -|- 2a;, 5 -|- 2a;, 

6 -|- 2a;, 7 -|- 2a;, 8 -|- 2a;, 1 -|- 3a;, 2 -|- 3a;, 3 -|- 3a;, 4 -|- 3a;, 5 -|- 3a;, 6 -|- 3a;, 7 -|- 3a;, 

8 + 3a;, 1 + 4a;, 2 + 4a;, 3 + 4a;, 4 + 4a;, 5 + 4a;, 6 + 4a;, 7 + 4a;, 1 + 5a;, 2 + 5a;, 

3 “h 5a;, 4 4- 5a;, 5 4- 5a; 


13 


a;, 1 4“ 2 4~ 3 4~ 4 4~ 5 4~ 8 4~ 2a;, 1 4~ 2a;, 2 4~ 2a;, 3 4~ 2a;, 4 4~ 2a;, 

5 4“ 2a;, 6 4~ 2a;, 3a;, 1 4~ 3a;, 2 4~ 3a;, 3 4~ 3a;, 4 4~ 3a;, 5 4~ 3a;, 2 4~ 4a;, 4 4~ 4a; 


17 


a;, 1 4“ 2 4~ 3 4~ 4 4~ 5 4~ 2a;, 1 4~ 2a;, 2 4~ 2a;, 3 4~ 2a;, 4 4~ 2a;, 5 4~ 2a;, 

6 4“ 2a;, 8 4~ 2a;, 2 4~ 3a;, 3 4~ 3a;, 4 4~ 3a;, 5 4~ 3a;, 6 4~ 3a;, 2 4~ 4a; 


21 


a;, 1 4“ 2 4~ 3 4~ 4 4~ 2a;, 1 4~ 2a;, 2 4~ 2a;, 3 4~ 2a;, 4 4~ 2a;, 5 4~ 2a;, 6 4~ 2a;, 

3a;, 5 4~ 3a;, 2 4~ 4a; 


24 


1 4“ 2 4“ 3 4“ 4 4“ 5 4~ 6 4~ 9 4~ 1 4~ 2a;, 2 4~ 2a;, 3 4~ 2a;, 4 4~ 2a;, 

5 4“ 2a;, 6 4~ 2a;, 7 4~ 2a;, 8 4~ 2a;, 5 4~ 3a;, 6 4~ 3a; 


28 


1 4“ 3 4“ 4 4“ 5 4“ 6 4~ 8 4~ 1 4~ 2a;, 2 4~ 2a;, 3 4~ 2a;, 4 4~ 2a;, 5 4~ 2a;, 

6 4“ 2a;, 7 4~ 2a;, 8 4~ 2a;, 2 4~ 3a;, 4 4~ 3a;, 5 4~ 3a;, 6 4~ 3a; 


29 


a;, 1 4“ 2 4~ 3 4~ 4 4~ 6 4~ 2a;, 2 4~ 2a;, 3 4~ 2a;, 4 4~ 2a;, 5 4~ 2a;, 6 4~ 2a;, 

3 4- 3a; 


33 


a;, 1 4“ 2 4~ 3 4~ 4 4~ 4 4~ 2a;, 5 4~ 2a;, 6 4~ 2a;, 7 4~ 2a; 


37 


a;, 1 4“ 2 4~ 3 4~ 4 4~ 4 4~ 2a;, 5 4~ 2a;, 6 4~ 2a; 


40 


1 4“ 2 4“ 3 4“ 4 4“ 5 4~ 1 4~ 2a;, 2 4~ 2a;, 3 4~ 2a;, 4 4~ 2a;, 5 4~ 2a;, 6 4~ 2a;, 

7 4“ 2a;, 3 4~ 3a; 


41 


a;, 1 4“ 2 4~ 3 4~ 4 4~ 7 4~ 2 4~ 2a;, 4 4~ 2a;, 6 4~ 2a; 
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d 


Values of c 


d 


Values of c 


44 


2 -|- 3 -|- 4 -|- 5 -|- 6 -|- 

8 “h 4 “h 5 4- 2ct^, 6 2ct^, T 2ct^ 


53 


2 -|- 3 -|- 4: oj 


56 


1 -|- 2 -|- 3 -|- 4 -|- 5 -|- 

7 2 “h 2ct^, 4 “h 2ct^, 5 2ct^, 6 2ct^ 


57 


1 -|- 2 -|- 3 -|- 4 ^ 


60 


1 -|- 2 -|- 3 -|- 4 -|- 5 -|- 

6 + a;, 8 + a; 


61 


2 -|- 3 -|- 4 4~ ^ 


65 


2 -|- 3 -|- 4 -|- 




2 -|- 3 -|- 4 2 -|- ‘2u? 


73 


3 + U), 4: + U) 


76 


1 -|- 2 -|- 3 -|- 4 3 -|- 

6 “h 8 “h 1 “h ‘2u? 


77 


4 -|- tc, 7 -|- tc 


^3 


4 -|- tc, 74 -|- tc 


88 


3 -|- 4 -|- 5 -|- 




1 -|- tc, 4 -|- tc 


92 


2 -|- 3 -|- 4 -|- 5 -|- 6 -|- 

8 + a; 


101 


6 -|- tc 


104 


4-|-tc, 5-|-tc, 6-|-tc, 8-|-tc 




5 -|- 


113 


3 + ^,4 + ^ 


1^ 


4 + oj, 5 -|- tc 


124 


2 -|- 4 -|- 5 -|- 6 -|- 8 -|- 


1^ 


4 -|- tc, 6 -|- tc 


136 


4 + ^ 


1^ 


3 -|- 


140 


4-|-tc, 6-|-tc, 8-|-tc 


mi 


3 -|- 7 4~ ^ 


149 


4 + ^ 


im 


l-ftc, 4-|-tc, 6-|-tc 


161 




im 


4 -|- tc, 6 -|- tc 


184 


7 + ^ 


im 


4 -|- tc, 6 -|- tc 


201 


6 -|- tc 


mi 


6 -|- tc 


236 


4 + ^ 




4 + ^ 


284 


4 + ^ 


321 


1 + ^ 



We now give some insight into how much of the 2-part of the Brumer element 
is needed for the conjecture to be true (see the comment at the end of section 
4). First note that in all of our examples the Brumer element had a non-trivial 
2-part. This is not generally true (see example 1, p. 172 of [HI]), but it might be 
true for certain classes of situations. More precisely, we have 3 examples (0.8%) 
for which the 2-part is 2, 207 examples (54.6%) for which it is 2^, 123 examples 
(32.4%) for which it is 2®, 40 examples (10.6%) for which it is 2'^ and 6 examples 
(1.6%) for which it is 2®. In all examples, the full 2-part is not needed for the 
conjecture to be true. Even more striking is that in 324 examples (85.5%) only 
half or less than half of the 2-part is necessary and in 96 examples (25.3%) the 
full 2-part can be removed. The value of 2® (i.e. the part of the 2-part needed for 
the conjecture to be valid) was 1 for 96 examples (25.3%), 2 for 204 examples 
(53.8%), 2^ for 67 examples (17.7%), 2® for 11 examples (2.9%) and 2 ^ for 1 
example (0.3%). The values of 2®“® (i.e. the maximal part of the 2-part that can 
be removed) was 2 for 173 examples (45.7%), 2^ for 190 examples (50.1%) and 
2® for 16 examples (4.2%). 

The following tables list the ideal class groups of all helds li' considered. Each 
entry consists of two parts. The first part gives the invariant factor decomposition of 
an abelian group A in the form (ni, . . . , nr) where nj > 2 for all j and | Ui for 
1 < 1 < r. The group A has structure Zm x • • • x Zn.,,. The second part gives the 
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number of class groups isomorphic to A. Note that the smallest class number was 2 
and the largest was 10064. 



Occurrence of class groups with 1 invariant factor 



(2) 


9 


(5) 


2 


(10) 


lai 


(26) 


3 


(34) 


A 


(50) 


3 


(52) 


2 


(58) 


3 


(74) 


Dl 


(82) 


HI 


(106) 


HI 


(113) 


1 


(122) 


1 


(130) 




(136) 


1 


(146) 


1 


(148) 


1 


(170) 


1 


(178) 


1 


(194) 


1 


(202) 


1 


(212) 


1 


(226) 


1 


(250) 


1 


(274) 


1 


(338) 


1 


(340) 


1 


(346) 


1 


(388) 


1 


(410) 


1 


(466) 


1 


(530) 


1“ 


(562) 


1 


(650) 


T 


(692) 


1“ 


(794) 


1 


(1130) 


1 


(1604) 


1 


(1810) 


1 


(1930) 


1 


(2026) 


1 


(2722) 


1 


(5910) 


1 





Occurrence of class groups with 2 invariant factors 



(2,2) 


HI 


(4,2) 


HI 


(4,4) 


HI 


(6,6) 


HI 


(8,4) 


1 


(10,2) 


13 


(10,5) 


1 


(10, 10) 


1 


(12,6) 


1 


(12,12) 


1 


(16,8) 


1 


(20, 2) 


5 


(20,4) 


HI 


(20, 10) 


HI 


(22,22) 


1 


(24, 12) 


1 


(26, 2) 


HI 


(26, 13) 


1 


(28, 14) 


1 


(30,3) 


1 


(30, 30) 


1“ 


(34,2) 




(36, 18) 


1 


(40,4) 


1 


(50,2) 


HI 


(50, 10) 


1 


(52,2) 


HI 


(52,4) 


HI 


(52, 13) 


1 


(58, 2) 


1 


(60, 6) 


1 


(68, 2) 


1 


(70,7) 


1 


(70, 14) 


1 


(72,9) 


1 


(74, 2) 


2 


(78,3) 


HI 


(82, 2) 


HI 


(100, 2) 


T 


(100,4) 


T 


(100, 10) 


T 


(102,3) 


1 


(106, 2) 


1 


(116,2) 


4 


(130, 2) 


T 


(130, 10) 


HI 


(146,2) 


T 


(148,2) 


2 


(150,3) 


1 


(156,6) 


1 


(164, 2) 


HI 


(170, 2) 


1 


(178,2) 


4 


(200, 8) 


1 


(204, 2) 


T 


(212,2) 


1 


(218, 2) 


HI 


(226, 2) 


T 


(232,2) 


T 


(244, 2) 


2 


(260, 2) 


HI 


(296, 2) 


1 


(300, 6) 


1 


(338, 2) 


HI 


(340,2) 


HI 


(346,2) 


1 


(356, 2) 


1 


(370, 2) 


1 


(390, 2) 


1 


(390, 3) 


1 


(404, 2) 


HI 


(410,2) 


2 


(424, 2) 


1 


(452,2) 


1 


(482, 2) 


1 


(488, 2) 


1 


(500, 2) 


1 


(530, 2) 


1 


(580, 2) 


1 


(580, 4) 


1 


(596, 2) 


1 


(628, 2) 


T 


(772, 2) 


1 


(820, 2) 


1 


(822, 2) 


1 


(984, 4) 


1 


(1096, 2) 


1 


(1172,2) 


2 


(1220, 2) 


1 


(2180, 2) 


1 



Occurrence of class groups with 3 invariant factors 



(4,2,2) 


HI 


(4,4, 2) 


OI 


(4,4,4) 


HI 


(8,4,2) 


HI 


(10,2,2) 


3 


(10, 10,2) 


1 


(12,6,2) 


OI 


(16,8, 2) 


1 


(16,8,4) 


1 


(16,8,8) 


1 


(18, 18,2) 


1 


(20,2,2) 


m 


(20,4, 2) 


HI 


(20,4,4) 


1 


(20, 10, 2) 


2 


(20,20,2) 


HI 


(20,20,4) 


1 


(24, 12,2) 


1 


(26,2,2) 


HI 


(28, 14, 2) 


3 


(34,2,2) 


HI 


(40,2,2) 


1 


(40,4, 2) 


T 


(40,4,4) 


3 


(40,8,2) 


1 


(40,20,2) 


1 


(44,44, 2) 


1 


(50,2,2) 


HI 


(52,2,2) 


HI 


(52,4,2) 


2 


(58,2,2) 


1 


(60, 12, 2) 


1 


(60, 12,6) 


1 


(68,2,2) 


4 


(68,4,2) 


1 


(100,2,2) 


1“ 


(100,4,2) 


OI 


(100, 10, 2) 


T 


(104,2,2) 


HI 


(104,4,4) 


1 


(104,8,4) 


1 


(106,2,2) 


OI 


(116,2,2) 


HI 


(120, 12,2) 


1 


(130,2,2) 


1 


(148,2,2) 


HI 


(178,2,2) 


1 


(194,2,2) 


1 


(202,2,2) 


1 


(244,2,2) 


1 


(250,2,2) 


1 


(260,2,2) 


1 


(274,2,2) 


1 


(290,2,2) 


T 


(292,4,2) 


1 


(340,2,2) 


HI 


(340,4,2) 


1 


(404,2,2) 


1 


(520,2,2) 


1 


(596,2,2) 


1 


(740,2,2) 


1 


(1830, 2, 2) 


1 


(2516,2,2) 


1 
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Occurrence of class groups with 4 invariant factors 



(6, 6, 2, 2) 


1 


(8, 8, 2, 2) 


2 


(20,2,2,2) 


1 


(20,4,2,2) 


1 


(20,4,4,2) 


1 


(20, 10,2,2) 


1 


(36,36,2,2) 


1 


(52,4,2,2) 


1 


(58,2,2,2) 


1 


(68,2,2,2) 


1 


(68,4,2,2) 


1“ 


(82,2,2,2) 


1 


(100,4,2,2) 


1 


(104,2,2,2) 


1 


(116,4,2,2) 


1 


(122,2,2,2) 


1 


(148,2,2,2) 


1 


(200,4,2,2) 


1 





Occurrence of class groups with 5 invariant factors 
(10,2,2,2,2) I 1 II (20,2,2,2,2) | 1 || (68,4,2,2,2) | 1 

Final note and acknowledgements. After having completed the full verification 
of the Brumer-Stark conjecture for all 379 examples listed here, Greither verified 
for us that all of our extensions are “nice” in the technical sense defined in his 
paper [G] . This makes our study of the 2-part of the Brumer element especially 
interesting (see comments at the end of section 1). We would like to thank 
Cornelius Greither for his help and we would also like to thank Igor Schein for 
helping us verify some of the most difficult examples. 
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Abstract. We outline a general algorithm for computing an explicit 
model over a number field of any curve of genus 2 whose (unpolarized) 
Jacobian is isomorphic to the product of two ehiptic curves with CM by 
the same order in an imaginary quadratic held. We give the details and 
some examples for the case where the order has prime discriminant and 
class number one. 



1 Motivation 

Let El , E 2 be two elliptic curves defined over Q with complex multiplication by 
an order O of an imaginary quadratic field K . We are interested in finding explicit 
models for curves C defined over Q whose (unpolarized) Jacobian is isomorphic 
to El X E' 2 - In this paper we propose a general algorithm for this purpose and 
give details only for the following special case where we have have carried them 
out: El — E 2 , O is the ring of integers of K — W = 3 mod 4 prime 

and O has class number one. Our special case consists of finitely many curves, 
up to isomorphism; the algorithm produces models over K for them. 

It is a general fact due to Narasimhan and Nori [NN] that there are only 
finitely many principal polarizations on a given abelian variety up to isomor- 
phism. Hence, for a fixed O there are only finitely many isomorphism classes of 
the curves we want; their number was calculated by Hayashida and Nishi [HN]. 

For a similar question in the case of abelian surfaces with complex multipli- 
cation by a quartic field see [vW]. 

Our interest in this problem arose in connection with a generalization to 
genus 2 of the singular moduli formulae of Gross and Zagier [GZ] for the norm 
of the difference of j-values of CM elliptic curves. (This generalization will be 
the subject of a separate publication.) As an illustration, consider the genus 2 
curve C determined by 



— f[x) — 6 ^h{x)E{x), 

* Much of this work was done while I was a guest at the Max Planck Institut fur 
Mathematik in Bonn in 1995. I take the opportunity to thank everybody at the 
Institut for their hospitahty. I would also hke to thank the NSF, TARP, and the 
Alfred P. Sloan Foundation for their generous support. 

W. Bosnia (Ed.): ANTS-IV, LNCS 1838, pp. 505—513, 2000. 

Springer- Verlag Berlin Heidelberg 2000 
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where 



h{x) = (7144y=TM - 151790);r3 -g (129789V=TM + 1752597);r2 

-h(-4748iy=TM + 510153);r + (-1596^=1^ - 37250) , 



and 

h‘'{x} — x^h{—l/x) 

(bar denoting complex conjugation of the coelhcients) . The unpolarized Jacobian 
of C is isomorphic over Q to the product of two elliptic curves with CM by the 
ring of integers of K — (Q(V— 163). Let 

D = 2-^2 (/) = (2 . 3^ • 5 • 7 • 11 • 17 • 19 • 23)^^ . 

Then we have 




where 

p4 4 6 \ 

Q[m) — i 4 55 1 j m 
V 6 1 83 / 

is a certain positive dehnite ternary quadratic form of level 163 associated to 
C and the (hnite) sum is over m £ Z® such that (163 — Q(m))/4 is a positive 
integer. In particular every rational prime I dividing D is smaller than 163/4 
and inert in K. 

The signihcance of the number D is that C has bad reduction only at primes 
dividing D. Note that over Q the Jacobian of C has good reduction everywhere 
but C does not; at primes diving D, C reduces to two elliptic curves crossing at 
a point. 

Another source of interest in the problem is the fact, which I learned from K. 
Lauter, that the reduction of the curves C provides genus 2 curves over certain 
hnite helds with maximal number of rational points (see §5 for an example). 
In this regard, the more interesting problem is the analogous one for curves of 
genus 3 for which we hope to exhibit in the near future an algorithm similar to 
the one sketched here. 

2 Outline of the Algorithm 

We start by giving an outline of the main steps of the general algorithm and 
then give details for our special case in the next sections. 

Step 1. Find period matrices for the polarized Jacobians. 

Step 2. Given a non-split period matrix obtained in step 1 compute a model 
for the corresponding curve. 
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The first step is purely algebraic and only requires computations with rational 
numbers; it involves the calculation of representatives for ideal classes of certain 
orders in a quaternion algebra (see [HN] for more details). What we need to do 
is describe explicitly the finitely many principal polarizations on E\ x E 2 up to 
equivalence. 

The second step relies on the following explicit version of Torelli’s theorem 
for curves of genus 2 due to Bolza and Klein. Let 

7^2 = {-Z" e I Z* — Z, lm.{Z) positive definite} 

be the Siegel upper-half space of rank 2. Let Z £ 7^2 be a period matrix of 
a principally polarized abelian surface which is not the product of two elliptic 
curves with the product polarization. A theorem of Torelli guarantees that Z 
arises from a curve of genus 2, unique up to isomorphism. Here is a way of 
recovering the curve from the period matrix Z . 

Let fz{ui, U 2 ) G C[wi , W 2 ] be the leading term in the Taylor expansion of 

9^^1,{U,Z) , U-{UI,U’2) 

odd 

about the origin, where for /U, iz £ {0, 1} 

z) := ^ ^ y ^ ^2) £ C 2 , Z e ?f2 

is the theta function with characteristics (see [Mu]). Then the canonically po- 
larized Jacobian of the hyperelliptic curve over C determined by the equation 

= fz{x,l) 

corresponds to Z . (There are six theta functions with odd characteristics and 
hence fz is a sextic, i.e. homogeneous of degree 6.) 

The difficulty in applying this formula is to know how to normalize the sex- 
tic fz properly to guarantee that its coefficients are algebraic integers as well 
as finding similar expressions for its Galois conjugates. In general, this would be 
accomplished by an application of Shimura’s general reciprocity law. We would 
obtain rapidly convergent series giving the minimal polynomials of these coef- 
ficients. Since the coefficients of the minimal polynomials of the coefficients of 
the sextic are in Z, truncating the series would then allow us to compute them 
exactly. We show how this works for our special case in §4. 

3 Principal Polarizations 

From now on we assume that O is the ring of integers of K — Et{\/—N) C C, 
A = 3 mod 4 prime and the class number of C? is 1. Hence, E\ — E 2 — E, with 
E isomorphic to C/ O over C. 
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The principal polarizations of T x T up to isomorphism correspond to positive 
dehnite unimodular Hermitian forms of rank 2 over O up to GL2(C^)-equivalence. 
In order to hud a set of representatives of these Hermitian forms we will exploit 
the happy accident that since we assume N = i mod 4 the quaternion algebra 
B — ( (up to isomorphism the unique quaternion algebra over Q ramihed 
only at N and oo) contains Q(i). This allows us to convert the question to that 
of hnding Hermitian forms over Z[i] of discriminant — up to equivalence and 
this is quite simple. Here is how it works. 

Consider in B the order 

R — 1j-\- TLi TL—^\ j) -|- /Zi— (1 -|- j), = —1, = —N. 

i? is a maximal order in B with a natural embedding of O sending \J —N to 
j. The rank 2 unimodular Hermitian forms arising from polarizations of E x E 
correspond to rank 1 left /^-modules. 

Since R also has an embedding of Z[i] (sending i to i) we may associate to 
a left i?-module a rank 2 Hermitian form <l> over Z[i]. We can give as a triple 
(a, b, c) with a, c £ Z>o and b £ E[i], where <P(u, v) — 2auu + buv + buv + 2cvv. 
It is not hard to see that this form has discriminant bb — 4ac = —N. 

It will be more convenient to work with SL 2 rather than GL 2 equivalence and 
to avoid duplications we consider only forms <l> — [a, b, c) with 6=1 mod 2. The 
above discussion establishes a 1-1 correspondence between principal polarizations 
on E X E, up to 5T2(G)-equivalence, and positive dehnite binary Hermitian 
forms <l> — [a, b, c) over Z[i] of discriminant —JV, up to 5T2(Zl[*])-equivalence. 

Let H be the hyperbolic 3-space 

M = {w = (a?, j/, f) G I f > 0} ! 

which we will think as embedded in the Hamilton quaternion algebra R by 
(x, i/,t) X + iy + jt (here i, j are the usual basis of H with R — p — —1 and 

£ SL 2 (C) then it is not hard to check that 
w 1—7- {aw + b){cw -|- 

sends H to H dehning an action of 5^2 (Q on H and in particular, an action of 
SL 2 {E[i\). This last action has a very simple fundamental domain, whose closure 
is given by w = {x, y,t) £ H with x'^ + y^ + > 1, x < 1/2, y < 1/2, f) <x + y. 

We can associate to a form ^ the point w — {b + \/N j)/ 2a £ H and we call 
<l> reduced if w lies in the fundamental domain. The action of SL2(/Z[i]) on H 
mimics that on Hermitian forms. Every form is SL 2 (/Z[i] (-equivalent to a unique 
reduced form. 

The situation is in fact very analogous to that of positive dehnite binary 
quadratic forms over Z and, as in that case, it is easy to write an algorithm that 
lists all reduced forms <d> of a. given discriminant (we do not really need N to be 
prime or class number 1 for this). Here is a brief sketch (all of this is classical 
going back to Hermite [He, I, p. 251]). 



U = -/*')• If 



c d 
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Input : N — i mod 4 

For 0 < r, s < \JN j2, r odd, s even 
Set m (r^ + + iV)/4 

For a|m, max(r, s) <a< ^/m 
Add [a,r + is,m/a) to List 
Add {a,—r + si,m/a) to List unless 
a — raja or r — a or s — a or s = 0 

Output : List 



As an example, we give in table 1 the list of reduced forms of discriminant —163. 



Table 1. Reduced Hermitian forms over Z[i] of discriminant —163 



(1,1,41) 

( 2, 1 + 2i, 21 ) 

(3, ±1 + 2i, 14) 
(6, ±1 + 2i, 7) 
(4, ±3 + 2i, 11) 
(6, ±5 + 2i, 8) 
(5,±l + 4i,9) 
(7, ±5 + 6^•, 8) 



In general, the number of <^’s is the class number n of B [Ei], which can be 
given in terms of N as follows (a formula valid for any prime A = 3 mod 4) 

[ n(^ + 5), iw) - +1 

n — < 

[^{N+U}, (^)=-l. 

Finally, given a = (a, b, c) as above, b — r + si, the matrix 




is a period matrix corresponding to the associated principal polarization on 

ExE. 

4 Bolza-Klein Sextics 

The product polarization on E x E corresponds to the reduced form <l> — 
(1, 1, {N + l)/4) in the principal class; hence, forms <l> not in the principal class 
correspond to curves. 
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Given a form <l> — [a, b, c) not in the principal class we define the associated 
normalized Bolza-Klein sextic as follows. 

where rj is Dedekind’s eta function. It satishes the following properties. 

— The 51^2 (G) class of depends only on the SL 2 {'Z,[i]} class of 

— has coelhcients in K and has coelhcients in O. 

— The Igusa invariants [Ig] of are in Z and depend only on the 51^2 (2i [in- 
equivalence class of <l>. 

The genus 2 curve 

C'# : 

is then dehned over K and, over the algebraic closure K of K in C, its Jacobian 
is isomorphic to E x E. 

Given a form tl> — [a,b,c) let <!>^ [a,—b,c). Suppose both tl> and <!>^ are 

reduced. Then tl> and <!>^ are not 5T2(2i[i])-equivalent but they are (always) 
GT2(2i[i])-equi valent. The corresponding curves are hence isomorphic 

over K; note that they are also complex conjugates of each other. Otherwise, 
curves G# corresponding to different reduced forms are non-isomorphic. The 
involution l has a natural counterpart on the left i?-ideals in B and it turns out 
that the number of orbits of i is what is classically known as the type number 
of B [Ei]. Hence, there are t — 1 isomorphism classes of curves with Jacobian 
isomorphic to Ex E, where t is the type number of the quaternion algebra B 
[HN]. 

Here is a table with the values of n and t for the primes N we are considering. 



Table 2. Type and class number of the quaternion algebra B 



N 


n 


t 


3 


T 


T 


7 


1 


1 


11 


2 


2 


19 


2 


2 


43 


4 


3 


67 


6 


4 


163 


14 


8 



Note that for N = 3 or 7 we only have the product polarization and hence 
there is no curve G with unpolarized Jacobian isomorphic to E x E m. that case. 

Given a curve G dehned over Q its field of moduli is the held E CQ charac- 
terized by the property: For every r £ Gal{Q/Q), G’’ is isomorphic to G if and 
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only if T is the identity on F. Clearly isomorphic curves have the same field of 
moduli. Notice that F is the smallest field over which a curve isomorphic to C 
could be defined, but it is not in general a field over which it can be defined. In 
fact, for example, Shimura showed that no generic hyperelliptic curve of even 
genus has a model over its field of moduli [Sh Thm 3] . See [Me] for a discussion 
of this issue for curves of genus 2. 

For the curves (7# the field of moduli is Q (the field generated by the Igusa 
invariants [Ig]), but, in fact, most are not definable over Q; only those forms 
which are 5L2(2i[*])-equivalent to <!>^ give rise to curves definable over Q. 

To see this we note that by their very construction the period matrices Z,^ lies 
in a certain real 3-dimensional cycle in %2 considered by Shimura [Shj. Namely, 
the cycle defined by 



Z (EU'i, 



0 1 

-1 0 



Z^-Z 



0 1 

-1 0 



If Az is the complex abelian surface corresponding to such a Z then there is an 
isomorphism 

A : Az — > Az, with A o A = —id. 

(Applied to Z<^ this yields the fact that the curves (7# and (7#t are both isomor- 
phic and complex conjugate to each other as mentioned above). 

It follows that if Az has no automorphisms other than ±id then it has no 
model dehned over its held of moduli. It is not hard to see that this holds for 
Z 4 >, for every in the interior of the fundamental domain. 



5 Examples 

We end with an illustration of the above discussion, giving the outcome of algo- 
rithm when A = 43. The calculations were done using PARI-GP. The routines 
as well as the data for all cases is available at: 
http : / / WWW . ma . utexas . edu/ users/ villegas 

The reduced forms of discriminant —43 are (1,1,11), (2,1 -|- 21,6) and 
(3, ±1 + 21, 4). 

1) For <^=(2,1 + 21, 6) we obtain 

fi^{x,l) — ”'/~43) x'^ 2 V~43) x“^ -\~ I) 

Its Igusa invariants are 

/2 = 1728012 

/4 = 93313728006 

/6 = -186622271996 

■h = -2176943579975806271997 

7io = 2176782336000000000000 
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(these were calculated using classical algorithms for invariants of a sextic follow- 
ing Mestre [Me]). 

As in the example of the introduction D — J\o — 2“^^ disc (/$) factors nicely 

£> = (22. 3. 5)^^ 

This curve descends to Q; here is a model 

t/2 = 24384 + 61311 x"^ + 585856 -f 813483 + 3214656 a? -f 1472877 . 



2) For ^ — (3,1 + 2i, 4) we obtain 

U(x, 1) = ^((14^=43 - 160) -7(42^=43 -f 162) ir® + 

(2247^^ - 159) -717021 

(2247^=43 -7 159) ;r2 +(-42^=43 -7 162) 

-714^=43 -7 160) 

Its Igusa invariants are 

J 2 = 14333772 

/4 = 7393823156166 

Je = 3726840435157546564 

Js = -312234946681873274015037 

/lo = 7355827511386641000000000000 

and 

£) = /lo = (2 • 3 • 5 • 7)^2_ 

As explained in §4, since <l> corresponds to a point in the interior of the 
fundamental domain the curve (7# has no model over Q. Alternatively, we can 
see this following Mestre [Me], When the curve has no extra automorphisms (i.e. 
its only automorphisms are the identity and the hyperelliptic involution), the 
obstruction to the curve being dehnable over its held of moduli (Q in our case) 
is given by a conic in 

xMx* = 0, X — (xi : X 2 : x^) £ P^, 

where M is a 3 x 3 symmetric matrix whose entries are certain invariants of the 
sextic; more precisely, the curve is dehnable over its held of moduli if and only 
if the conic has a rational point there. Explicitly, we have M — (niij) with (we 
have actually simplihed slightly the matrix given by Mestre) 

mil = ~ I 6 O/ 4/2 — 3600/e 

TJI 21 — — P4P2 “f 330PcP2 + I6OJ4 

msi = — JqJ2 — 840p6d4 — 8 OOOP 10 

m 22 = — 2576^2 — 8 P 4 P 2 — 120p6d4 — 2 OOOP 10 ’ 

^71-32 = Q7JqJ4, + 6OO/10P2 + 90/g 

m 33 = — 33 Pgp 2 — 100PgJ4 — 8OOP10P4 
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where J 2 , J 4 , Jq, Js, Jio are the Igusa invariants. 

In our case we have 

mil = -21538723388574481387776 

mi2 = 24856361223852137345176064256 

TO13 = -23971255400369899892885589544571136 

m22 = -28732882146400381994651008552571136 

TO23 = 27776672840855638207256856144392139100416 

m33 = -26987491534155851141341724256178812956900004096 

We easily verify that this conic has rational points everywhere locally except at 
the primes 43 and 00 ; in particular, it has no rational points. 

We should point out that the vanishing of the determinant of M precisely cor- 
responds to the curve having extra automorphisms. As with D, this determinant 
factors nicely 



det M = -2®"^ • 3^® • 5®"^ • 7^® • 19"^ • 29^ • 37^ • 43 . 

Finally, let p be a prime which splits in K — Q(V— 43) as p — W. The 
reduction of the curve (7# modulo V gives a smooth curve C of genus 2 over 
Fp. We have verified that for all primes in the range 167 < p < 10000 such that 
4p = a? + 43 for some a £ N the curve C or its quadratic twist attains the 
maximum number of points possible, namely p -|- 1 -|- 2 [^2p ] (an improvement 
on Weil bounds due to Serre). 
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Abstract. This paper analyzes reduction of fractional ideals in a purely 
cubic function field of unit rank one. The algorithm is used for generating 
all the reduced principal fractional ideals in the field, thereby finding 
the fundamental unit or the regulator, as well as computing a reduced 
fractional ideal equivalent to a given nonreduced one. It is known how 
many reduction steps are required to achieve either of these tasks, but 
not how much time and storage each reduction step takes. Here, we 
investigate the complexity of a reduction step, the precision required in 
the approximation of the infinite power series that occur throughout the 
algorithm, and the size of the quantities involved. 



1 Introduction and Motivation 

Basis reduction of fractional ideals is one of the key ingredients in the compu- 
tation of invariants of a purely cubic function field of unit rank one, such as the 
fundamental unit, the regulator, the ideal class number and, most importantly, 
the order of the Jacobian of the field. In fields of characteristic at least five, a 
basis reduction procedure was first presented in [2], and its discussion was con- 
tinued in [1]. The algorithm was originally used for generating the entirety of 
reduced fractional principal ideals and thus finding the fundamental unit and the 
regulator of the field. Unfortunately, there are usually exponentially many such 
ideals, and enumerating them all is not the most efficient method for computing 
the regulator. This is where another aspect of ideal basis reduction comes into 
play: it quickly produces from a given nonreduced fractional ideal an equivalent 
reduced one. 

The infrastructure of the set of reduced principal ideals is a powerful tool for 
invariant computations and a variety of other applications in both computational 
number theory and cryptography. Loosely speaking, the product of two reduced 
fractional principal ideals is generally not reduced; however, reduction produces 
a reduced ideal “close to” the product ideal, and the number of basis reduction 
steps required is polynomial in the size of the field. This phenomenon can be 
exploited for computing invariants of the field much faster than with the naive 
approach outlined above. For hyperelliptic, i.e. quadratic function fields (where 
reduction amounts to computing a simple continued fraction expansion), this 
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was successfully accomplished in [3] with an improvement in complexity from p 
to essentially where p is the number of reduced fractional principal ideals. 
Work on the purely cubic setting is in progress at the time of writing, and we 
expect a similarly dramatic speed-up from our original method of [2] . 

While it is known how many reduction steps are required to compute the 
fundamental unit and the regulator of a purely cubic function field of unit rank 
one and characteristic at least five, it is as yet unclear how long an individ- 
ual reduction step takes, how large the inputs and outputs get, and how much 
“precision” is required. Numerical experiments and heuristics suggest that the 
answers to these three questions are ‘not very long’, ‘not very large’, and ‘not 
too much’, respectively — at least in the reduced case — but we lack proof. This 
paper remedies this rather unsatisfactory situation. To that extent, we provide 
answers to the following questions: 

• What is the complexity of an ideal basis reduction step? 

• What is the size of the quantities involved? 

• What is the minimal precision required in the approximation of the infinite 
series involved? 

2 Purely Cubic Function Fields 

A detailed treatment of this material can be found in [2] and [1]. A purely cubic 
function field is the function field of a plane curve given by the (not necessarily 
nonsingular) model — D{x) =0 over a finite field fc = of order q whose 
characteristic is not 3; here, D{x) G k[x\ is a cubefree polynomial. Thus, a purely 
cubic function field can be viewed as a cubic extension K = k{x){p) of a rational 
function field k{x) obtained by adjoining a cube root p of a cubefree polynomial 
D = D{x) G k[x]] this makes it the function field analogue of a purely cubic 
number field. We write D = GH^ where G,H G k[x] are squarefree and coprime 
and deg(G) > deg{H). 

The integral closure O of k[x] in K is both a ring and a fc [a;] -module of 
rank 3 that is generated by the integral basis {l,p, w} where cv = /H, so to 

is a cube root of D = G^H. If a = a + bp + cojGK {a,b,c G k{x)), then 
the conjugates of a are of = a + bop + cifui and a" = a + bf^p + cluj where 
t is a fixed primitive cube root of unity. The norm of a is N(a) = aa'a" = 
+ b^GH^ + c^G^H - SabcGH G k{x). 

We henceforth make the following assumptions: 

• q = —1 (mod 3) (so k contains no primitive cube roots of unity), 

• deg(D) = 0 (mod 3), 

• The leading coefficient sgn(D) of D is a cube in fc* = fc \ {0}. 

Then K/k{x) has two points at infinity, namely one rational point and one 
quadratic point. The former gives rise to an embedding of K into the field 
k{x~^) of Puiseux series over k, and the Galois closure of K is embedable into 
k{L){x~^); nonzero elements in k{x~^) (respectively, k{L){x~^)) have the form 
^ with Qi G k (respectively, fc(t)) for i > —m 

and a-m 0. The degree valuation on k(x) extends canonically to k{x~^) via 
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deg(of) = m and to k{i){x via deg(a + (ii) = (deg(a + + /3i^)/2 = 

deg(of^ — aj3 + /3^)/2 G Z (a,/3 G k{x~^)). For a = ^ k\x~^), 

we set |a| = sgn(a) = a_m, and [aj = (with |0| = 0 and 

[OJ = 0). For a € K, we have \a'\ = Note that |G| > \H\ 

implies \p\ < |w|. 

Under the above assumptions, K has unit rank 1 over k{x)] that is, the the 
group O* of units in O is isomorphic to fc* x Z (see Theorem 2.1 of [2]). A 
generator e of the torsionfree part of O* is a fundamental unit of K/k{x). If 
e has positive degree (and is hence unique up to constant factors), then R = 
deg(e)/2 = — deg(e') is the regulator of K/k{x). 

3 Fractional Ideals 

A fractional ideal (of O) is a subset f of X such that there exists a nonzero 
d G k[x] such that df is an integral ideal in O, i.e. an additive subgroup of O 
that is also closed under multiplication by elements of O. The unique monic 
polynomial d = d(f) of minimal degree that satisfies this condition is the denom- 
inator oi f. f is principal if it consists of O-multiples of some 6 G K; write f = (6). 
The fractional ideals form an infinite Abelian group I under multiplication, of 
which the set of principal fractional ideals forms an infinite subgroup V. The 
factor group J/V is the ideal class group of K/k{x); it is a finite Abelian group 
whose order is the ideal class number of K/k{x). The product h = Rh' where R 
is the regulator of A'/fc(a;) is the order of the group of fc-rational points on the 
Jacobian oi K; it is independent of the element x and thus the representation 
of df as a function field. Two fractional ideals are equivalent if lie in the same 
coset in I/V, i.e. if they differ by a factor that is a principal fractional ideal. 

We will henceforth assume “fractional ideal” to mean “nonzero fractional 
ideal containing 1” . Then every fractional ideal f is a fc[a;]-module of rank 3 with a 
basis {1, /i, v}] write f = [1, p,,v].li f = [1, /i, v] where = (mo + mip + rri 2 Uj)/d, 
v= (no + u-ip + ri 2 w)/d with mo, mi, m 2 , no, ni, n 2 , d G k[x] jointly coprime 
and d = d(f), then the norm of f is A^(f) = a(min 2 — m 2 ni)/d^ G k{x) where 
a G fc* is chosen so that N{f) is monic. The discriminant of f is 

1 1 1 

p. p' p" I G k{x). 
n n' n'7 

Both A^(f) and A(f) (up to a constant factor) are independent of the choice of 
fc[a;]-basis of f, and A^(f) is multiplicative on the set of fractional ideals. 

A canonical basis of a fractional ideal f is a fc [a;] -basis {l,a, /3} where a = 
s'{u p)/s,P=s"{v wp w)/s with s, s', s", u, u, w G fc[a;], s' s" divides 
s, s" divides H, and gcd(s',id) = 1. Here s = d(f) up to sign, and we may 
assume |s'u|, |s"u| < |s|, and |w| < |s'|. Such a basis always exists, and it is a 
simple matter to generate a canonical basis from any given basis, or compute 
a canonical basis of the product ideal of two fractional ideals given in terms of 
respective canonical bases (see [1]). 
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An element 0 in a fractional ideal f is a minimum in f if for any (/) G f, 
\(f)\ < \9\ and \(f)'\ < \0'\ imply (j) G kO] that is, 4> differs from 0 only by a constant 
factor, f is reduced if 1 is a minimum in f. It is easy to see that an element 0 is a 
minimum in O if and only if the fractional principal ideal f = {9~^) is reduced. 

We summarize some properties of fractional ideals; the proofs of these results 
can be found in [2] and [1]. 

Proposition 3.1. Let f be a fractional ideal. 

1. Z\(f) = afN{tjYA for some a G k* . 

2. \d{f)\-^<\Nm<m\-\ 

3. Iff is reduced, then |A(f)| > 1, so |IV(f)| > 

4- Iff is reduced, then |d(f)| < so |IV(f)| < |A||c?(f)|“^. 

5. //|Z\(f)| > |d(f)P, i.e. |d(f)| < |fV(f)||Z\|^/^, then f is reduced. 

6. Iff is nonreduced, then |IV(f)| < so |A(f)| < 

Let f be a fractional ideal and let 0 be a minimum in f. An element (/) G f is 
the neighbor of 6 in f <p is also a minimum in f, \9\ < and for no ■0 G f, 
\9\ < 101 < 101 and |0'| < \9'\. 0 always exists and is unique up to nonzero 
constant factors (see Theorem 5.1 of [2]). 

The Voronoi chain of successive minima in O where 9\ = 1 and 

9n+i is the neighbor of in O yields the entirety of minima in O of nonnegative 
degree (Voronoi first investigated this chain in cubic number fields in [4]). This 
chain is given by the recurrence 6*„+i = where fin is the neighbor of 1 in 

the reduced fractional principal ideal f„ = [9~^) {n G N). The first nontrivial 
unit e = 0p+i (p G N) encountered in this chain is the fundamental unit of 
K of nonnegative degree. Since the recurrence for the Voronoi chain implies 
dmp+n = for m G Nq and n G N, {fi, (21 ■ ■ ■ , fp} is the complete set of 

reduced principal fractional ideals in K. The positive integer p is the period of e. 
By Theorem 6.5 of [2], p = so there may be (and usually are) 

exponentially many reduced fractional ideals in K/kfx). 

4 Reduced Bases 

For the remainder of the paper, we exclude the case of even characteristic, so k 
has characteristic at least 5. For 9 = 1 + mp + nuj G K with l,m,n G k{x), we 
define 

= 9 — I = mp + noj, 

rje = {1 + 2i)~^{9' — 9") = mp — nuj, (4.1) 

Ce = 9' + 9” =21 — mp — noj, 

where k) is a primitive cube root of unity. Then 

^ = 2 fo + Ce)) (4-2) 



so 



\9'\ = max{|? 7 e|, |Ce|} 



\^e\ < max{|6»|, \0'\} 



(4.3) 
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If {1, 6*, (/)} is a basis of a fractional ideal f, then 

(4-4) 

A fc[a;]-basis {1, /i, v} of a (reduced or nonreduced) fractional ideal f is reduced if 

ICaiI < 1) ICl^l ^ Ij I^All < 1 ^ /a r'l 

if |? 7 i,| = 1, then \v\^l. \ ■ ) 

The following procedure (which is essentially Algorithm 7.1 in [2]) generates a 
reduced basis of a fractional ideal. 



Algorithm 4.1. (Ideal Basis Reduction) 

Input: fl, V where {1,(1, v} is a basis of some fractional ideal f. 
Output: pL, V where {l,pL,v{ is a reduced basis off. 

Algorithm: 

1. Set p, = (1, V = V. 

O’’ \Vu\ < \Viy\, replace 




by 





3- If |?7ax| > \Vu\ 

3.1. While > |A(f)|^/^, replace 



by 



0 1 
-1 



3.2. Replace 



by 



3.3. If\'n^\ = \'nv\, replace 



0 1 
-1 



by 



1 -sgn{T]f,T],,^)\ ( p 
0 1 



4 . While |? 7 i^| < 1, replace 



by 



0 1 
-1 



While \pfi\ > 1, replace 



by 



1 0 j Ur 
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5. Replace p, by p, — [C/iJ/2 and v by v — [Cj^J/2. 

6- If |?i/| = hi/| = 1; replace v by v - [v\. 

A reduced basis provides an easy means by which to determine whether or not 
an ideal is reduced (see [1]): 

Proposition 4.2. Let {1, ^, 1 ^} be a reduced basis of a fractional ideal f. 

1. If f is reduced, then p is the neighbor of 1 in f. 

2. f is reduced if and only if\p\ > 1 and max{|i/|, |? 7 i^|} > 1. 

3. f is nonreduced if and only if |/i| < 1 or \v\ < \r]i,\ = 1. 

Part 2 of Proposition 4.2 in conjunction with (4.3) and step 5 of Algorithm 4.1 
implies that step 6 can only be entered if the input ideal f is nonreduced. Part 1 
of this proposition together with the recursion for the Voronoi chain shows that 
repeated application of Algorithm 4.1 to the ideal f„ = (0~^) with subsequent 
division of f„ by the neighbor p„ of 1 in f„ generates all the minima of nonnega- 
tive degree in O and hence the fundamental unit of K. A similar recursion allows 
for computing from a given nonreduced fractional ideal an equivalent reduced 
one. 

Let f be any nonreduced fractional ideal and define a sequence (f„)„g|^ of 
pairwise equivalent fractional ideals as follows. 

fi = f, fn+i = where (()„ = | ^^ ^ (n € N) (4.6) 

and {1, pn, Vn\ is a reduced basis of f„. The case 4>n = Vn in (4.6) can happen at 
most once; that is, if f„ is nonreduced with |/i„| > 1, then is reduced and a 
reduced basis of can be obtained directly without applying Algorithm 4.1: 

Proposition 4.3. Let f be a nonreduced ideal with a reduced basis {1, p,v} and 
let g = = [1, pi^~^, If \l\ > then g is reduced with a reduced basis 

Proof. If|^| > 1, then by part 3 of Proposition 4.2 and (4.3) \v\<l = \pi,\ = \v'\. 
Let a = pv~^ and j3 = v~^. Then \a'\ = \p'\ < 1, so \rja\ < 1, |Cq| < 1? ^nd 
since |a| > 1, |^a| = |o:| by (4.2). Furthermore, |/3'| = 1, so \(^p\ < 1, and 
l^/sl = \P\ = \v\~^ > 1- Since pp = , \pp\ = 1. 

Since |a| > 1 and max{|/3|, \pp\} > 1, g is reduced by part 2 of Proposition 
4.2. Since |^a| = |a| > \v\~^ = l^/sl, |?7a| < 1 = h/?!. ICal < 1. and |C/3| < 1, 
{1, a, /?} is a reduced basis of g. 

A polynomial number of steps of recursion (4.6) produces a reduced ideal (see [1]): 

Proposition 4.4. Let f = fi be a nonreduced fractional ideal. 

1. The recursion (4.6) produces a reduced fractional ideal equivalent to f for 
some m gN. 
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2. If m in part 1 is minimal, i.e. is reduced and f„ is nonreduced for n < m, 
then 

m < maxjl, ^ (^5 - deg(fV(f)) - ideg(Z\)^| . 

3. If f is the product of two reduced ideals and m is as in part 2, then 

TO < ^ (deg(Z\) + 4) . 

As an aside, we mention the infrastructure of the set {fi, f 2 ) ■ ■ ■ ) fp} of reduced 
principal fractional ideals. If fj = for i = 1,2, . . .,p, then the distance of 

fi is 5i = deg(0i). From part 3 of Proposition 4.4, a reduced principal ideal f can 
be obtained by applying no more than 3(deg(Z\) +4)/8 iterations of (4.6) to the 
initial (generally nonreduced) product ideal f^f^. Moreover, <5(f) = 5i + 5j + 5 
with 5 = 0(deg(Z\)) = O(logp), so the distance of f is within a logarithmically 
small ‘error’ of where one would expect it to be. As pointed out in section 1, 
this phenomenon allows for much faster computation of the fundamental unit 
and other invariants of K/k{x). 

The implementation of Algorithm 4.1 raises a number of questions: How large 
do the degrees of 9, and rjg {6 G {fi, i^}) and those of their basis coefficients 
get throughout the algorithm? How often the while loops in steps 3.1 and 4 
executed? And how does one determine absolute values of fg and rjg, and compute 
the quantities in steps 3.2 and 4 as well as [? 7 p/? 7 i/J in step 4? These 

questions will be addressed in the next three sections. 

5 Input/Output Sizes in Ideal Basis Reduction 

We begin with the following empirical observation; for quadratic integers (as 
opposed to Puiseux series), this is referred to as the Gaufi-Kuz’min law. Let 
a = oo C k{x~^) and define Oj = G k[x] and «i+i = (oi — ai)~^ for i G Nq. 
Then the Oi (z G No) are the partial quotients in the simple continued fraction 
expansion of a, and for z G N, Oi will almost always have very small degree. The 
quotients in steps 3.1, 3.2, and the first while loop of step 4 are easily 

seen to be partial quotients in the simple continued fraction expansion of 
where /zq and vq are the inputs of step 3.1 or, if that loop is never entered, of 
step 3.2; similarly for in the second while loop of step 4. These quotients 

will therefore almost always have very small degree, with the possible exception 
of the very first such partial quotient. 

Let {l,fi,i'}he a reduced basis of some fractional ideal f that was computed 
using Algorithm 4.1. Since |? 7 ^| < 1 < \r]i,\, and rj^, and 77 ^ differ by a factor 
that is a partial quotient as described above, |? 7 i^| will usually have quite small 
degree, and |? 7 ^| will not be much smaller that 1. By (4.5) and (4.4), |^j,| < 
so usually, will be close to |Z\(f)|^/^, and since and fi, 
once again differ by a factor that is a partial quotient in a simple continued 
fraction expansion, |^j/| will not be much smaller than 

We have the following rigorous bounds on reduced bases: 
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Proposition 5.1. Let he a reduced basis of a fractional ideal, where 

/i = (mo + mip + m2Wi)/d, v = (no + n\p + n2oS)ld with mo, m\, m2, no, n\, n2 
G k[x] and d = d{f). 

1. [mo/c?J = \mip/d\ = \m2ijj/d\ = 3[^J. 

2. 1^1 < max{9“MZ\(f)|i/2}, |mo|, |mip|, |m 2 w| < max{gr“ \v\ < 

maxjl, gr“^|Z\(f)|^/2}, Imp + n 2 w| < |no| < max{|d|, 

3. If \p\ > 1, then \v\ < \p\ < |Z\(f)|^/2, |mo| = |mip| = |m 2 w| < 

|no|, |nip|, |n 2 w| < 

Proof. Part 1 follows immediately from < 1 and |C^| < 1. For part 2, 
we note that from (4.2), (4.5), and (4.4) |p| < max{|^^|, |^^|} with < 1 
and = |Z\(f)|^/^|? 7 i,|“^ < |Z\(f)|^/^. The bounds on |mip| and |m 2 w| follow 
from |mip — m 2 w| = |d? 7 ^| < |d| and |mip + m 2 w| = < |dfV(f)||Z\|^/^ < 

by (4.4) and the first two parts of Proposition 3.1. Furthermore, |mo| < 
max{|d^i.|, |(iCi/|}- Now by (4.2) |i^| < max{|Cp|, with |Ci.| < 1 and < 
\ffi\ < l^(f)l^/^ by (4.4), \df,,\ < \dff,\ < and |no| < max{|dCi.|, \dfu\} < 

max{|d|, ( 7 “^|Z\|^/^}. The bounds in part 3 follow from part 1, (4.2), and the fact 
that |d| < by part 4 of Proposition 3.1. 



Note that unfortunately, we have no rigorous upper bound on Ipi^l and hence 
on |nip| and |n 2 w| in the (nonreduced) case where |p| < 1. However, as we saw 
above, these values will generally not be too large. We proceed to analyze the 
sizes of the inputs of step 2 of Algorithm 4.1. 

Lemma 5.2. 

1. Let he a fractional ideal and let where {l,/i„,i/„} is a 

reduced basis of {n G N). Let f = = [1, for some n G N 

with p = pn and 1 / = i/n- Then 



max 





1 



wy 



1 



- min{|p|,|p'|}’ i| <max<{ \ 



IpI’Ip'IJ min{|p|, |p'|}' 



If is reduced, then 



max{|? 7 ^-.i|, ^ 



max{|mp-i|, 
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If f„ is nonreduced and is the product of two reduced fractional ideals, then 
max||77^-i|, I < 

2. Let {l,a,/3} be a canonical basis of a fractional ideal f. Then 



P 

d{f) 



< |?a|, \Va\ < \P\, 



(jJ 



< max{|^^|, |? 70 |} < |w|. 



Iff is reduced, then |^a|, |?7a| > |w| \ max{|^^|, \rjp\} > \p\ ^ 
If f is the product of two reduced fractional ideals, then 

2 2 
\U,M > |^|?/2|^p max{|^^|,|r;0|}> 



Proof 1. By (4.3) |? 7 ^-i| < \p'\ \ < max{|^| M^'| < 

W'\\p'\~^, and < max{|i/||^j-i|, Since \v\ < max{l, |^^|} 

< maxjl, |^|}, we have < max{l, |^|“^} < \v'\ max{|^|“^, 

A simple computation reveals that 






P'P" 






Pufti - VfiCu 
2p'p" 



Since max{|Cp|, |? 7 ^|} = one of rj^-i and Pufi-^/Pu has absolute value 

Now by (4.4) \ = \A(f)\^/'^, so |A(f)|i/2 cannot exceed 

both summands in absolute value. By (4.4), an upper bound on the absolute 
values of both terms is given by 



W\ ^ ^ I ^.fX|i/2 max{|^|, l^'ll 



since A(f) = N{p) M(f„). 

If f„ is reduced, then |^| > 1, so = max{|^|, |^'|} = |^| and \v\ < 
maxjl, I ^ 1 ^ 1 } < |/i|. Furthermore, 



A Cfi ^ Cfj-Ci' ^PtJ.Pi' 

If IC^I = Im'I. then = W'\\p'\~^ and = \2p~^ - C^-i I = 

If \Pu\ = Im'I, then |?7^-i| = \p'\~^ and = \2np~^ - C^-i I = 
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Wm~^. Finally, = |iV(M)llM|-' > \N{^,)\\A{U-^/^ = and 

WW\-^ = \A{U^/^M~^ < |Z\(fJ|i/2|7v(^)| = |zi(f)|i/ 2 . 

If f„ is nonreduced, then |^| < 1. If is the product of two reduced ide- 
als, then by part 3 of Proposition 3.1, |^(f„)| > ^(fi)| > Then 

W? > |^(Ai)| = |/\(f)"M(f„)|i/2 > q^\A{f)A\-^/^ and min{|^|, |^'|} > 
q-^\N{fi)\>q^\A{f)A\-^/\ 

2. Let a = s~^s'(u + p), /3 = s~^s"(v + wp + w)}. Then = Va = s's~^p, 
= s” s~^{wp + oj), pp = s” s~^{wp — oj) with |w| < |s'|. Since |s's"| < 
Isl = M(f)l IpI < |w|, the first set of bounds follows. If f is reduced, then 
|(i(f)| < = |po;| by part 4 of Proposition 3.1. If f is the product of two 

reduced fractional ideals, then |c?(f)| < |IV(f)|“^ < q‘^\A\ by parts 2 and 3 of 
Proposition 3.1. 

We point out that in the situation where Algorithm 4.1 is applied to the prod- 
uct f of two reduced fractional ideals (as is the case in the infrastructure scenario, 
for example), the input is a canonical basis and not of the form 

We now proceed to investigate the workings of ideal basis reduction in more 
detail; in particular, we will see how the sizes of the quantities 77 ^, and 

Pi, change throughout Algorithm 4.1. We point out that after step 2 of the 
algorithm, and |? 7 j,| < |? 7 ^|. 

Lemma 5.3. 

1. In step 3.1 of Algorithm 4-1, £,f_i and p^ do not increase in absolute value 

in the first iteration and decrease in absolute value in each subsequent iter- 
ation. ^ 1 / and pi, decrease in absolute value in each iteration. Furthermore, 
l'?Ai| > I’ImI ^ \3^\ each iteration. 

2. Step 3.2 of Algorithm 4-1 decreases and p^, but does not decrease pi, 

in absolute value. After execution, > |^j/| and \p^\ < \pf\- 

3. Step 3.3 of Algorithm 4-1 leaves the absolute values of fi,, and pi, un- 
changed, but decreases Pf,, in absolute value. After execution, > |^j/| and 

\'nu\ < M- 

Proof. Let {a, /?} be the input and {p, v} the output of any iteration of step 3.1, 
step 3.2, or step 3.3. 

Since \fi,pi,\ > |A(f)|^/^ if and only if - Pf_i/pi,\ < 1, or equivalently, if 

and only if \ have in step 3.1 






Vu = 










Pv = -Pa 



P^ 

.m. 



VP- 



Therefore |^j,| < |^/3| = and |?7i,| < |?7^| = |?7^|. From step 2 of the algorithm, 
in the first iteration |^q| > and |?7 q| > |?7^|, so < |^a| and |?7^| < |?7a|. In 
subsequent iterations, we have |^q| > |^/3| and |?7 q| > |?7^|, so = |^/3| < |^q| 

and |? 7 ^| = |? 7 ^| < |? 7 a|. 
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In step 3.2, the transformations on are the same as in step 3.1, so 

each of these quantities decrease in absolute value, and we still have 
Furthermore, 




The first term in the difference has absolute value less than |? 7 ^|, while the 
second term is at least |? 7 ^| in absolute value because |[?7a/?7/3j ~ L^a/'C/sJI — 1- 
So > |?7^| = |?7^|. 

In step 3.3, we have v = (3, so and rjn are unchanged. Furthermore, if 
a = sgn(? 7 a? 7 ^^), then = \^a ~ a^/sl = l?ol as \^a\ > l^/sl = Finally, since 
a = [Va/V!3\, < h/sl = M- 

Analogous results hold for step 4 of Algorithm 4.1: 

Lemma 5.4. 

1. In the first loop of step 4 of Algorithm 4-1, and decrease in absolute 
value in each iteration, while and rji, increase in absolute value in each 
iteration. 

2. In the seconds loop of step 4 of Algorithm 4-1, and increase in absolute 
value in each iteration, while and rj^, decrease in absolute value in each 
iteration. 

3. Throughout step 4> 1'Ca‘I ^ \3v\ ^ I'di'l- most one of the while 

loops in step 4 is entered, and after the last iteration of either of the loops, 

o.nd \■q^f\ < 1 < \pu\. 

The previous two lemmata show that \rjv\ takes on its largest value through- 
out the algorithm either after step 3.2 or after the first loop of step 4 if that 
value is less than 1 after step 3.2. Since in both cases = |A(f)|^/^, and 

we generally at least expect > |<i(f)|“^, we usually have by parts 1 and 2 of 
Proposition 3.1 \rj,y\ < |d||A(f)|^/^ < for this maximal value. 

6 Complexity of Ideal Basis Reduction 

We now investigate how often each of the while loops in the basis reduction 
algorithm. 

Proposition 6.1. Let f = [l,pL,v] where pi,v are the inputs of Algorithm 4-1. 
Assume that > |CI a'nd |? 7 ^| > \r],y\, so step 2 has been executed. Denote by 
r, s, and t the number of iterations of step 3.1, the first loop in step 4> and the 
second loop in step 4> respectively. Then 

r < max ■|o, — 

r -I- s < maxjO, deg(C) - ^ deg(A(f))}, r -|- t < max{0, deg(? 7 i.) -|- 1}. 



deg(C^7^^) - 2 1 
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Proof. Let {^j,o,vq} be the first input and {^i,Vi} the output after iteration i 
(1 < z < r) of step 3.1. From part 1 of Lemma 5.3 and \r|,y^\ < 

I’ii/i-il? so inductively and \r|,y^\ < for 1 < z < r. Then 

Again, let {/io, vq} be the first input and {/ii, Vi} the output after iteration 
f (1 < f < s) of the first loop of step 4. Then \rj^g\ < 1. Analogous to the 
previous part, we infer from Lemma 5.4 that \rjvi\ > 9*|^7i/ol for 1 < z < s, and 
|?7i/s-il < 1 < hi/J- Then 1 > q\rj^^_^\ > q’"\ri^o\. Here, vq is the v value output by 
step 3.3 and hence by 3.2 (since 3.3 leaves it unchanged). Thus, the corresponding 
rji, is the quantity rjur+n where we interpret step 3.2 as the (r + l)-st iteration 
of the loop in step 3.1. Now \'qvr+i\ = ^ Thus, 

In the second loop of step 4 of Algorithm 4.1, we have |? 7 ^J < q *|? 7 ^o| for 
1 < z < f, and |? 7 ^J < 1 < Then 1 < where |^o| is 

/i value output by step 3.3. The corresponding \rjfj_g \ is at most equal to 
Then = \iq^^\ < q~'^\'qu\, so < q\iq^\. 

Corollary 6.2. Let r, s, and t he as in Lemma 5.2. Let f he the input ideal and 
{1,^, iz} the input basis of Algorithm 4-1. Assume that\ffj_\ > |^j/| and\r]^\ > \r]i,\, 
so step 2 has been executed. 

1. Suppose f = for some zz S N, where fi is a fractional ideal, = 

with {l,^„,zz„} a reduced basis of f„ {n € N). 

If f„ is reduced, then r = s = 0, ^ deg(A(f)) + 1. 

If f„ is nonreduced and is the product of two reduced fractional ideals, then 
?■ < ^deg(Z\) - i, r + s < ideg(A) - 3, r + f < i deg(Z\(f)Z\) + 1. 

2. Suppose {1,^, iz} is a canonical basis off. Then 

f’ < ^(deg(d(f)) + l), r+s < max{0,deg(d(f))-deg(w)}, r+t < deg(p) + l. 

If f = O, i.e. V = p and p = to, then r = s = 0, t < deg(p) + 1 . 

If f is reduced, then ^ ^ deg(A), r + s < deg(p), r + t < deg(p) + 1. 

If f is the product of two reduced fractional ideals, then 

’’<^deg(A), r + s < i deg(A) + deg(p) - 2, r + t < deg(p) + 1. 

Proof. Part 1 follows directly from the bounds in Lemma 5.2. For part 2, let 
{ 1 , 0 ,/?} be a canonical basis of f with a = s's~^p and /? = s” s~^ {wp + oo). 
Since IpI < |o.|, |^„z;„||A(f)|-i/2 < |sVI|s"o.|-i < |s| = |d(f)|, |?a||A(f)|-i/2 = 
|s||s"w| ^ < |d(f)||a;| and \rja\ < \p\- Once again by Proposition 3.1, |d(f)| < 

I if f is reduced and |d(f)| < q~“^\A\ if f is the product of two reduced ideals. 

If iz = p and p = to, then \fi,riv\ = |pP < = |A(0)|^/^ and |^j/| < 
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Corollary 6.2 reveals that if the input ideal f of Algorithm 4.1 is either equal 
to O (with basis {l,p,oj}), or is of the form f = where f„ is 

reduced and is a reduced basis of f„, then step 3.1, the first while 

loop in step 4, and step 6 can be omitted. This is the case, in particular, if the 
regulator or the fundamental unit of K/k{x) are computed by generating the 
recursion with = fp+i = O. 

Algorithm 6.3. (Basis Reduction, Input Ideal of Special Form) 

Input: ft, V where is a basis of some fractional ideal f. Here, = 

{p, w} or {p,, v\ = where {1, 4>, 6} is a reduced basis of a reduced 

fractional ideal. 

Output: p, V where {l,p,v} is a reduced basis off. 

Algorithm: 

1. Set p = p, V = V. 

O’’ |?7p| < \p„\, replace 



by 



3- If > \vu\ 
3.1. Replace 



by 



3.2. If\Pfj.\ = \pu\, replace 



4 . While |? 7 p| > 1, replace 



by 



by 



0 1\/m 

-1 oj Ur 



0 1 
-1 Lr/rjA^U 



1 -sgn{pf,p,,^)\ ( p 
0 1 



1 0 j Ur 



5 . Replace p by p — LCaiJ/2 v by v — [rj/2. 



7 Precision Required for Ideal Basis Reduction 

When computing absolute values as well as integer parts of quotients as required 
in basis reduction algorithm, the relevant quantities of the form bp ± clo need to 
be approximated to sufficient “precision” with a Puiseux series in k{x~^) that is 
truncated at some suitable negative power of x. Our numerical experiments in 
[2] show that increasing the precision or even using variable precision does not 
have a significant impact on the running time of the algorithm; for example, a 
reduction in precision from deg(H) to deg(H)/2 made a difference of only 5-10 
percent in computation time. Nevertheless, it is desirable to have a lower bound 
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on the minimal precision required; in [2], where we implemented Algorithm 4.1 
for reduced ideals only, we relied exclusively on heuristics and numerical evidence 
in determining our precision. 

We define a relative approximation of precision n € No to an element a = 
€ k{x~'^) to be d„ = aiX~\ Then |1 - d/a| < or 

equivalently, |a — d| < q’o approximate a quantity of the form 6 = 

bp + coj with 6, c G k{x), such as rj^, and r],^, we generate relative approxi- 

mations pn and Cjn of sufficient precision ntop and to, respectively, and approx- 
imate 6* by 6* = bpn + cCjn- Pn is precomputed by explicitly extracting a cube root 
of H G k[x] so that the coefficients of . . . , x, 1 , x~^, . . . , j;"-deg(£')/3 

correct, and a)„ is given by the following lemma. 

Lemma 7.1. Let be a relative approximation of precision n to p. Then = 
|^^n-deg(oj)^2 /^j^deg(oj)-n ^ relative approximation of precision n to uj. 

Here, it is a simple matter to verify that |1 — < q~"‘. Henceforth, we 

denote by /5„ and relative approximations of some precision n G N to p and 
to, respectively. For 6 = a + bp + av with a, &, c G k{x), we set 

9 = a + bpn + CUn, ie=bpn+CUn, m = bpn - cQjn, fs = 2a - bpn ~ CUJn- 

The following lemma gives lower bounds on the precision required to compute 
absolute values and integer parts of certain Puiseux series correctly. 

Lemma 7.2. Let 9,(f> G k{x). 

1. IfmGl and > max{|^e|, \rie\}, then |^g| = q™ if and only i/||e| = 9™; 

l^el < g"* if and only if ||e| < g"*, and |^e| < g"" if and only if ||e| < g"". 

2. Lfq^ > maxjl, ^ |, then |^e| = ||e|- 

3. Lf q^ > max{|^e|, \pe\}, then \ 9\ = [0J and [CeJ = L6J- 

4- If |?e| > |^,^| and g" > maxll, ^ 1 , then ^ . 

[ 80 80 80 J l80j [^0 

Proof. If 9 = a + bp + cuj with a, &, c G k[x], then 

- ^1 = = ICe - Cel 

= |6(p - Pn) + c{uj -0Jn)\ < max{|6p|, |cw|}g“” = max{|Ce|, |pe|}g“”. 
This immediately yields parts 1-3. For part 4, we have 

Ce ^ _l_ Ce(C0 - C0) Ce ~ Ce 

C0 C0C0 C0 

Suppose that |Ce| > IC0I and g" > max{l, |Ce/C0l, |ge/C0l, ICe?70/C0l}- Then 

<Mmax{|C 0 |,|p 0 |}g-"<l; 

similarly, |(|e - Ce)/C0l < 1- So [Ce/C0j = LCe/C0j- 
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We are now able to give lower bounds on n for the different steps of Algorithm 

4.1. We consider a precision of n to be sufficient if in any identity or condition on 

a quantity 9, 9 can be replaced by a relative approximation 9 of precision n to 

9. For example, n is sufficient for step 3.1 of Algorithm 4.1 if 

exactly if and if in every iteration of the 

loop. 



Lemma 7.3. Let f he the input ideal and the input basis of Algorithm 

4-1 Define {a, (3} = {7,(5} = (m, such that |,Jq,| > and \r]~f\ > \rjs\- Let r, 
s, and t be as in Proposition 6.1. Then a precision of n is sufficient for 

I?- 



1. step 2 and the if condition at the start of step 3 if > max 



ih 



2. step 3. 1 if q"' > max 

I 



/ 










fh_ 


1 


?/3 


5 




5 


id 



1^9? 1.-2 \m\^ 

5 Q 



where = 






for r > 1 ; 



3. step 3.2 if q^ > max • 



4 . step 3.3 if q'^ > max} 1 







Ip 




•Ca 'qs 


id 


5 


id 


5 


S/3 



|A(f)|l/ 2 ’|Z\(f)|l/ 2 ’" A(f)|l /2 



^ 27-2 \vs\^ 









|A(f)|l/2/’ 



5. the first while loop of step 4 ifq^ > max <j |A(f)|^/^, \f,f}\,q\ 



r.21,-1-2 



where 



max < 


im 


\ and q^“-^ = 


— 1 


0<2<S-1 


im 


/ 





6. the second while loop of step 4 if q'^ niax jg'", g'"* | where q^ 

and ( 7 ™* = 



max 

0<j<t-l 









9m 



1. steps 5 and 6 if q^ > maxjg™*, 



Proof. We use the results of Lemma 7.2 and the same notation as in the proof 
of Proposition 6.1. We only prove parts 1-3 and part 5; the other parts follow 
analogously. 

1. Since g">max{l, Ipa/fa]}, |^?a| = ||a|, and since g” > max{|^^/,fa|, Ipp/fal}, 

l^i'l < \^u\ if and only if Finally, g" > ma,^{\ris/'ri-r\,\^s/'rij\} 

implies |? 7 ^| < \rj^\ if and only if |f}^| < \f]^\ and |? 7 ^| > \rj^\ if and only if 
\9u\ > \9’'\- 

2. We have a = j = p,Q and j3 = 5 = vo, < |^/ 3 |, \r|^^\ < \rjs\, and > 

|2\(f)|^/^ > Ifu.ilml for 0 < z < r - 1. Furthermore, = IVm/Vmly so 

\ = \m/^9\ = for 0 < z < r - 1. 




530 



Renate Scheidler 



Hence, since g" > max{l, |^a/?77l}. and for 

0 < i < r — 1. Also, if r > 1, then 





- fh 


7l/r7l/r-l 


<u-. \m\^ 


U 


— Q 


1 

-t 

1 


|A(f)|iA’ 



so I ^1,^1 = Furthermore, since 

then < | A(f)|^/^/|^i.,| if and only if < | for 0 < z < r. 

Finally, if r > i > 1, then < |^^77i|/| A(f)|i/2 

and Also, for 0 < z < r- 

1. Hence, g" ^ max{|^„/,f,3|, \rij/^i3\, |^/3?75|/| A(f)|i/2, Iz^aj V|^(f)|^/^} implies 
= L4 ,/|i.J for 0 < z < r - 1. 

3. If r > 1, then = q^'' , 







- 




\3S\^ 




U 


— H 


1 

1 


^ |A(f)|iA’ 


£/ir7l/r 1 


- 


Vl^rVl^r-l 


/ 27-2 




2 

Ur 


— Q 


(Arv 

-t 

1 

-t 

1 





5. We have \iq^^\ < 1 < so > |A(f)|i/2 > for 0 < z < s - 1. 
Since g” > max{l, > max{|?7i,J, |^j,J : 0 < z < s — 1}, \r|^^\ < 1 if and 
only if \f/^^\ < 1. Also < |A(f)|^/^, so g" > max{l, |A(f)|^/^} yields 

Now I < for 0 < z < s — 1, and for 0 < z < s — 2: 



3m 




3vi 


1 


^Ui3vi 


|A(f)|i/2 ^ 1 




<v 




|A(f)|iA’ 


Si/,j 





and 



Vfia-l 



nt^S-1 —2 



r,ls — l — 3 



< 



< 



- |A(f)|l/2’ 



Vua-l^fJ.a-1 



Si/,, 



\A^/^ 
l£i/,-J 



= q 



21 . 









It follows that L£zii/£i/iJ = [im/iiyi\ for 0 < z < s - 1. 



Corollary 7.4. Let f be the input ideal and {1, /i, zz} the input basis of Algorithm 
4-1 or Algorithm 6.3. Define {«,/?} = {7,(5} = |/i, zzj such that |£q| > |£/ 3 | and 
1^7 1 > \vs\- Let I, Ir, Is-i, rn, and rrit be as in Lemma 7.3. 
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\m\" 



Ml [> ^^671 a precision of n is sufficient for 
Algorithm 4-1. 



2. If g” > max • 



/ 






ll_ 








ialS 


1 


?/3 


5 


ip 


5 


h 


5 


^P 



|Zi(f)|i/2 

precision of n is sufficient for Algorithm 6.3. 



JM 



— , then 



We point out that the values , q^‘~^ , q^, and g'"* are almost always very 
small. In general, we expect the case where f is the product of two reduced 
ideals to require the highest precision, since in this case, |fV(f)|“^ (and hence 
the upper bound on |c?(f)| by part 2 of Proposition 3.1) is largest. Even in this 
situation, it is very likely that the required precision is not too large, say no 
more than deg(Z\); however, only numerical experiments will tell. The scenario 
of Algorithm 6.3 requires significantly less precision: here, we expect deg(A)/2 
to be sufficient, and this bound is supported by numerical evidence (see [2]). 



8 Conclusion and Outlook 

We have provided a complete analysis of the algorithm for computing a reduced 
basis of a fractional ideal in a purely cubic function field of unit rank 1. The 
number of iterations of each while loop of the algorithm is bounded by a fraction 
of deg(Z\). The quantities |^^|, |CI) and \rj^\ appear not to grow too large 
throughout our computations; in fact, we expect the bounds of Lemma 5.2 to 
significantly exceed the actual sizes of these quantities. Finally, the precision 
required to compute absolute values and quotients appears to be a fraction of 
deg(Z\) as well. 

As mentioned in section 1, our two algorithms serve two purposes. If Algo- 
rithm 6.3 is repeatedly applied, starting and terminating with f = O, it generates 
all the reduced principal fractional ideals in O and thus produces the fundamen- 
tal unit and/or the regulator of K/k{x) as illustrated in [2]. Algorithm 4.1 can be 
used to determine from a given nonreduced fractional ideal an equivalent reduced 
one. In particular, if the input ideal is the product of two reduced principal ideals, 
then the infrastructure of the set of reduced fractional principal ideals guaran- 
tees that the method finds a reduced principal fractional ideal “close” to the 
product ideal very quickly, namely after at most 3(deg(Z\) -|-4)/8 applications of 
Algorithm 4.1. This phenomenon allows for a rapid movement through this set, 
thereby speeding up regulator and fundamental unit computation significantly. 
The technique can be extended to yield the ideal class number of K/k{x) and 
hence the order of the group of fc-rational points on the Jacobian of K. Work on 
this problem is currently in progress. 

li q = —1 (mod 3), then a representation of unit rank 1 can always be 
achieved for any purely cubic extension K/k{x) by applying a simple change 
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of variable; in particular, any purely cubic extension of unit rank 0 (i.e. when 
deg(I?) is not a multiple of 3) can always be converted to one of unit rank 1 
over the same field of rational functions k{x). The methods outlined above can 
also undoubtedly be generalized to arbitrary cubic function fields of unit rank 1; 
once again, this is currently being explored. In addition, we are in the process of 
investigating the case of even characteristic. It remains to be seen which elements 
of Algorithms 4.1 and 6.3 (if any) are of use in cubic extensions of unit rank 2, 
and to what extent our techniques can be extended to unit rank 1 extensions 
of degree higher than 3. Much of the reduction theory remains valid here, but 
Algorithm 4.1 needs to be replaced by an entirely different reduction procedure. 
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Abstract. Let O be a maximal arithmetic in one of the four (non- 
split) composition algebras over R, and let [p] = mn be the norm of 
an element p in O. Rehm [14] describes an algorithm for finding all 
factorizations of p as p = af3, where [a] = m, [f3] = n and {m,n) = 1. 
Here, we extend the algorithm to general p, m, and n, providing precise 
geometrical conhgurations for the sets of left- and right-hand divisors. 



1 Introduction 

A composition algebra over M for the bilinear form 

[x,y] = Xiyi H \- Xnyn, 

where we define the (squared) norm of an element x as [a;] = [a;,a;], is a not- 
necessarily-associative division algebra satisfying the composition law 

[xy] = [a;] [y] 

for all x,y G M”. Hurwitz [8] proved that composition algebras occur only in 
dimensions n = 1, 2, 4, and 8; the algebras, unique up to isomorphism, consist of 
M, C, the quaternions H, and the octonions O. Frobenius [7] had earlier shown 
that M, C, and H are the only associative finite-dimensional division algebras 
over M. The octonions are not associative, but they are an alternative algebra, 
meaning that the left and right alternative laws 

x'^y = x{xy) and yx^ = {yx)x 

hold for all x,y G O. By a theorem of Artin (see [11]), any subalgebra of an 
alternative algebra generated by two elements is associative. 

Multiplication in the octonions can be described quite easily using the fol- 
lowing sets of coordinates. Let any element a; G O be an M-linear combination 
of eight orthogonal unit vectors 1 = ioo, io, *i, • • • , *6- For t = 0, . . . , 6, define 
multiplication among l,it,it+i,it +3 (subscripts taken mod 7) to coincide with 
the multiplication of the basis elements 1, i,j, k of the quaternions, H: 

= f = e = -l 

ij = k ji = —k 

* I wish to thank J. H. Conway and Princeton University for their support during my 
graduate work. The results of this paper are taken from my doctoral dissertation. 
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Since C can be defined as having basis elements 1 and i, we see the containments 
M C C C H c O. 

Each composition algebra C contains sets of elements that are arithmetics in 
the sense of Dickson [4] and Lamont [9] . An arithmetic A is a subset of C which 
contains 1; is closed under addition, subtraction, and multiplication; and is such 
that each element a G A satisfies [a] G Z and 2 [a, 1] G Z. A is said to be a 
maximal arithmetic if it is not contained in any other arithmetic of C. 

Define the naive arithmetic Nc as the one whose elements are simply Z-linear 
combinations of 1 and the imaginary units given above for C = C, HI and O; and 
let O represent any arithmetic of C containing Nc- In the remainder of this 
paper, the term arithmetic will refer only to such O. For C = M and C, O is the 
set of rational integers Z and Gaussian integers {a + bi \ a,b G Z}, respectively. 
For C = H, up to isomorphism there is one arithmetic properly containing TVh, 
namely Hurwitz’ integral quaternions 

{a + bi + cj + dk \ either a, b,c,dG Z, or a, 6, c, d G Z + -}. 

For C = O, there are four non-isomorphic O: Nq C C C [9], [15]. 
The maximal arithmetic can be described in terms of the coordinates given 
above for O: 

= {Oooioo + oo*o + • • • + 06^6 I each at G Z/2, and {at} H Z G S'}, 

where S consists of the subsets of {ooo, oq, . . . , aej whose indices are taken from 
{0, 0124, 0235, 0346, oo045, 0156, oo026, oo013| and the complements of these in- 
dices in oo0123456. Geometrically, is similar to the Es lattice [2]. 

The problem of finding the factorizations of a given p G O as p = a(3 for 
a, (3 G O and fixed m = [a] and n = [/3] has a long history. Factorization results 
for O in R, C, and El are classical (see [4]). However, the methods of associative 
number theory are not well-suited to O C O since, for instance, every one-sided 
ideal in is in fact two-sided and generated by a rational integer [1], [10]. 

We now summarize what is known for the four non-isomorphic O G O. 
Rankin [13], in a study of multiplicative functions, gives the number of factor- 
izations in No in the two cases (m, n) = 1 and m = p, n = where p is a 
prime. Pall and Taussky [12], using results of Estes and Pall [5] on the genera 
of certain octonary quadratic forms, determine the factorizations in Nq for gen- 
eral m and n. Feaux and Hardy [6] then extend their work to and O^. 

Unfortunately, these results, although “constructive,” do not lead readily to a 
geometric understanding of the sets of divisors. 

Recently Rehm [14] produced an algorithm that finds all factorizations of p 
in the maximal octonion arithmetic when (m, n) = 1. In this paper, we show 
that methods can be extended to general p, m, and n in the maximal arithmetic 
of any composition algebra, providing precise geometrical configurations for the 
sets of left- and right-hand divisors. 
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2 The Algorithm 

Let O be a maximal arithmetic of a composition algebra C containing fVc > and 
let Om C O consist of the elements of norm m. We start with p\ G Omomi, where 
mo > mi >0. We wish to find the set L^gipi) C Omo of left-hand divisors of 
Pi of norm mo- 

Define the conjugate a of an element a to be a = 2[a,l] — a. From the 
shape of Voronoi cell of O (see Coxeter [3] for O = C O), there exists a pair 
{7I) P2} C O such that 



. , [mi] m? 

Pi = limi + p 2 , where 0 < [P 2 J < = —■ 

mi divides [P 2 ], since mi divides every term on the right-hand side of 

[P 2 ] = [P 2 ] = [pi -71 wi] = [pi] -bm?[ 7 i] - mi( 2 [pi, 7 ij). 

Let m 2 be such that [P 2 ] = mim 2 . Then mi > ^ > m 2 > 0. If m 2 yf 0, we 
can repeat the arguments above with p 2 and m 2 , leading to pa and m 3 ; and 
so on. At some point we must reach an mN+i = 0, and thus pn+i = 0, since 
mi > mi+i > 0 for any m^ > 0 and i > 1. Thus, we obtain a finite collection of 
elements in O whose relationships are summarized in Figure 1. 



pi = 7imi -I- p2 
p2 = 72012 -I- p3 



[pi] = momi 
[p2] = mim2 



mo > mi 
mi > m 2 



pN-i = 7iv-imjv-i -f pjv 
pN = 'yNiriN 



[piv-i] = m]v- 2 m]v-i 
[Pn] = mN-iniN 



miv -2 > mjv-i 
ruN-i > mjv > 0 



Fig. 1. A “Euclidean algorithm” in O. 



Now, let Pat be any element of Omw Since any subalgebra of O generated 
by two elements is associative, we may write 

Pn = iNrriN = Jn{pnWn) = (7nPn)'Pn- 

Set pn -1 = 7nPn, so that pn-i is a left-hand divisor of pat of norm m^-i- 
Then pN-i is a right-hand divisor of both p]v and m^-i, and thus also of pn-i, 
since 

Pn- 1 = 7N-imN-i + 'pN = {7 n-iPn-i)pn-i + PnPn-i 

= {iN-lPN-l + Pn)PN-1- 

Setting pn -2 = 7n-iPn-i + Pn, we obtain a left-hand divisor of pn-i of norm 
mAT- 2 - We can continue this procedure until we arrive at a left-hand divisor po 
of Pi of norm mo- Figure 2 summarizes this process, which can be thought of as 
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[Pn] 


= mjv 


pN ~ PN-1PN 


Pn-1 = ')nPn 


i^iV-l] 


= rriN-i 


pN-1 ~ PN- 2 PN-I 


Pn-2 = Jn-iPn-i -I- Pn 


[pN- 2 ] 


= miv-2 


pN-2 = PN-3pN-2 


PN-3 = ')N-2PN-2 -I- Pn-1 


[tiiv- 3 ] 


= miv-3 


p2 = P 1 P 2 


pi = 'J2P2 -f P3 


[f'l] 


= mi 


pi = popT 


po = Jl pi -t p2 


[tie] 


= mo 



Fig. 2. Factoring the pj. 



tracing the left-hand side of Figure 1 from bottom to top, factoring along the 
way. 

We remark that the multiplication of the algebra is not used in an essential 
way in Figure 1. Moreover, products involving triples of elements in Figure 2 
occur only within associative subalgebras. 



3 The Configurations of the Divisor Sets 



We now determine Lmo(pi)- 

Lemma 1. Let 7 = aj3 = a' j3' , where [a] = [a'\ ^ 0 and [fi] = [f}'\ ^ 0. Then 
the angle 9a between a and a' is equal to the angle 6^ between f3 and (3' . 

Proof: Taking the inner product of 7 with aj3' , we obtain 

[a][/3, /3'] = [a(3,a(3'] = [ 7 , a/3'] = [a'/3',a/3'j = [a', a] [/?'], 



which yields 



cos 6 a 



[a, a'] 



IMl 

W] 



cos 9b. □ 



To initiate the procedure presented in Figure 2, take pN to be any member of 
OmN- Denoting geometrical similarity by we obtain the following sequence 
of similarities from Lemma 1 by alternatively setting 7 equal to pi and m, for 
appropriate z: 



OraN = {Tn} ~ {pN-l} ~ {TN-i} ~ ~ {pi} ~ {/io} = LmoiPl)- 

The final equality follows from Lemma 1 and the relationships among the pi, 
since distinct p € Lmoipi) correspond to distinct p' G Lmf^{pN) = Note 

that we also now know the set of right-hand divisors of pi of norm mi : Rmi (pi ) = 

m- 

We still have to determine mN- Let gcd(? 7 i , . . ,,r]k) denote the greatest com- 
mon rational integer divisor of 71 , . . . , in O. 
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Lemma 2. Let di = gcd{pi, rrii-i, rrii) for 1 < i < N . Then di = di+i for 
l<i< N. 

Proof: First, see that di \ pi+\ = pi — 'jirrii since di divides pi and rrii, so 
di I Pi+i- It divides rrii by definition. Finally, di \ rrii+i since di divides each term 
in the last line of 

Wi+i = = (;^)([pi] +mf[7i] -mi(2[7i,pi])) 

= rrii-i + mi[7i] ~ 2[7i, Pi]- 

Thus, di I di+\. By a similar argument, di+\ \ di as well, so di = di+\. □ Note 
that rriN = gcd(pAr, mAr_i, mA?) since uin divides both p^ and uin-i- Thus, 
Lemma 2 implies that uin = gcd(pi, mo, mi). 

In all of these discussions, we could just as well have computed the set of 
right-hand divisors of pi of norm mp. Thus, we conclude with 

Main Theorem. Let p G O have norm mn, and let d be the greatest common 
rational integer divisor of p, m, and n. Then the sets of right- and left-hand 
divisors of p of norm m are geometrically similar to Od- 
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Abstract. Let y) denote the number of integers < x that are com- 
posed entirely of primes bounded by y. We present an algorithm for 
estimating the value of 'l'{x, y) with a running time roughly proportional 
to y/y. Our algorithm is a modification of an algorithm by Hunter and 
Sorenson that is based on a theorem of Hildebrand and Tenenbaum. This 
previous algorithm ran in time roughly proportional to y. 



1 Introduction 

Let y) denote the number of integers < x that have prime divisors < y. The 
running times of many integer factoring and discrete logarithm algorithms make 
use of this function. As a number of important cryptography protocols rely on 
the difficulty of either integer factoring or the discrete logarithm problem, it is 
important for the security of such schemes to have good estimates for y) (see 
for example [10,13]). Mathematicians have studied the behavior of this function 
and obtained a number of estimates for it [5,6,7,8,11,12]. 

Until recently, the standard way to estimate y) computationally was to 
use the estimate 'l'{x,y) « a;p(log a;/ log y) [6], as Dickman’s function p{u) is 
relatively easy to compute. However, Hunter and Sorenson [9] showed that a 
theorem of Hildebrand and Tenenbaum [7] gives a much better approximation 
to y) in practice, and can be computed using a number of floating point op- 
erations that is roughly proportional to y. We refer to this method as Algorithm 
HT. 

In this paper, we show how to modify Algorithm HT to improve its running 
time to roughly ^ floating point operations. We pay for this drastic improve- 
ment in a slightly larger error, and we need to assume the Riemann Hypothesis to 
show the error is not excessive. We also present the results of some experiments 
comparing our improved Algorithm HT-fast to the original. 

Several other algorithms for estimating ^(x,y) deserve mention. Bernstein 
has presented an algorithm for computing 'P[x,y) exactly [3], and another algo- 
rithm that gives rigorous upper and lower bounds on 'P(x,y) [4]. Also, several 
other algorithms are mentioned in [9], although they are not original to that 
paper. 
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2 Background 

Before we discuss our improvements to Algorithm HT, we need to review some 
background material. We begin by reviewing Algorithm HT. 



2.1 Algorithm HT 

First, we introduce some notation and state the theorem upon which the algo- 
rithm is based. Define u = log a;/ logy. 

Let u := u{x, y) = minjlog x, y}/ logy = min{u, y/ logy}. 

Define 



C(s,y) := 

p<y 

(j){s,y) := logC(s,y); 

y) := y) {k > 1); 



HT{x,y, s) := 



x%{s,y) 



s^J2^T(|)2{s,y) 

Let a = a{x, y) be the unique solution to the equation 

(j)i{a,y) -I- log a; = 0. 



( 1 ) 



Theorem 1. We have 

<I'{x, y) = HT{x, y, a{x, y))(l + 0(l/u)) (2) 

uniformly for 2 < y < x. 

For the proof of this theorem, see Hildebrand and Tenenbaum [7]. 

Algorithm HT then proceeds as follows: 

1. Compute a list of primes up to y using a sieve (see, for example, [14]). 

2. Set ao := log(l -h y/(51oga;))/logy. 

3. Using oo as a starting point, find a solution a' to (1) via Newton’s method. 
Stop when \a — a'j < minjO.OOOl, l/(uloga;)}. 

4. Output iLT(a;, y, a'). 

In theory, in Step 3 above a preliminary search by bisection is required to guar- 
antee a running time of 0(y { + logiogy }) floating point operations. In 
practice, Newton’s method converges quite nicely after only a few iterations. In 
[9], it is proved that HT(x, y, a') = HT(x, y, a)(l -I- 0(l/u)). 
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In Steps 2 and 3, the following formulas are used: 



C(s,y) 

p<y 



Ms,y) 



-E 

P<V 



logp . 

p® — 1’ 



<p2{s,y) 



^ p^ilogp)^ 

(p" - 1)" ' 



We can now briefly explain what our improvements are to this algorithm. The 
idea is, instead of finding the primes up to y and using them to evaluate C, 4>i, 
and 4 > 2 , we only use the primes up to roughly and then approximate these 
functions using the prime number theorem. We will, however, need the Riemann 
Hypothesis in order to bound our error. 



2.2 Computation Model 

We measure the complexity of our algorithm by counting the number of floating 
point operations. Such operations include addition, subtraction, multiplication, 
division, comparisons, exponentiation, and taking logarithms of real numbers. 
We also include array indexing and branching as basic operations. In practice, 
we used 80-bit floating point numbers. 

2.3 Notation 

p always denotes a prime number, and sums over p are always sums over primes. 
tt(x) denotes the number of primes up to x. For positive functions / and g, 
we write f{n) = 0{g{n)) if there exists an absolute constant c > 0 such that 
f{n) < c- g{n) for all n sufficiently large. f{n) <C g{n) means f{n) = 0{g{n)). 



2.4 Approximating Sums of Primes 

We define 

r 1 

li(a;) := / ; dt 

Jo logt 

where the point at t = 1 is omitted. Let tt(x) = li(a;) -I- e{x) (the prime number 
theorem). By assuming the validity of the Riemann Hypothesis, we can take 
e{t) = 0{\/tlogt). We make frequent use of the following lemma. 



Lemma 2. Let f{p) he a eontinuously differentiable function on an open interval 
containing [2, oo), and let2 < z <y. Then we have 



E ^^P") = / + /(y)e(y) - f(z)e(z) 

z<p<y ^ 



e{t)f{t)dt. 



For details, see Bach and Shallit [2], Section 2.7 and Theorem 8.3.3. 
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3 Algorithm HT-Fast 

We begin by defining the following three functions: 





I exp 


[logy/fcj 


Vp<- ) 







[logy/fcJ , 

p<z k — 1 



C{s,y,z) := 



P^(logp)^ 2 logz ylogy 



p^z 



(p® — 1)^ s(z® — 1) s(y® — 1) 



1 



s(l — ks) 



[logy/fcj 

E 



( 1 + logy- 



1 



1 — fcs 



HTf{x,y,z,s) := 



- ( 1 + logz- 

x"A{s,y, z) 



1 



1 — fcs 



Sa/27tC'(s, y, z) 



To evaluate li(a:), we use standard techniques for the exponential integral: either 
5.1.55 or 5.1.11 (truncated) as appropriate from Abramowitz and Stegun [1]. 
The time to compute this is only 0(1) operations. Thus, given a list of primes 
up to z, the three functions A, B, and C can be computed in 0(7 t(z) + logy/s) 
operations. This is significantly smaller than the 0(7r(y)) operations to compute 
c, (j)i, or 4)2. 

Let (5 > 0, and assume that 5 log a; < Our new algorithm is as follows: 

1. Set z := min{y, max{1000, 5y^}}. 

2. Compute a list of primes up to z. 

3. Set «o := log(l + y/(51oga;))/logy. 

4. Using ao as a starting point, find a solution af to (1) via Newton’s method, 
substituting B{s,y,z) for —4>i{s,y). Stop when: 

\a — a/I < minjO. 000001, 0.1/(uloga;)} and | log a; — B{af,y, z)| < 1. 

5. Output iLT/(a;, y, z, a/). 

In the three theorems that follow, we show that our three new functions 
reasonably approximate C(s, y), 4‘i{s,y), and 4>2{s,y). After that, we show that 
the root a / found in Step 4 will in fact be a good approximation to a. 



Theorem 3. Let <5 > 0 such that 1 > s > 1/2 + <5, and let z ^ oo such that 
^ z < y. Assuming the validity of the Riemann Hypothesis, we have 

C(s, y) = A(s, y, z) ^1 + O ^ • 
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Proof. We have 

c{s,y)=i[{i-p-T^ n 



p^z 



z<p<y 






p<z 



\z<p<y 



Focusing on the sum inside the exponential, we have 

^ -iog(i-p-) = - 



z<p<y 



z<.p<y k — 1 



^—ks 



= E E 

k—1 z<p<y 

where we have used the expansion log(l + s) = 1^1 < 1- 

Next, we approximate using Lemma 2: 

, . ri~^^ ry f—ks II— ks ~—ks ry 

fE^~r=l IlSiT'" + —'<»>- —4-’> + / 40»r‘-‘<ir 



With our assumption that s > 1/2 + 5 and that e{t) = 0{\/tlogt), we can bound 
our three error terms by 0{logz/{kz^^)). Substituting v = the integral 

simplifies to (li(y^“''^) — li( 2 ^“''^))/fc. 

Finally, observe that we can truncate the infinite sum at fc = [(log y) / sj 
without any significant effect on the error, as the sum converges geometrically. 

□ 



Note that our requirement in the algorithm that 5 log a; < forces 1/2 + 

5 < QfOj which guarantees the condition on s during the algorithm, as we will 
always have ao < s. The requirement that s < 1 is not very restrictive, as 
Of < 1 + 0(1/ logy) was proven in Lemma 4.1 of [8]. 

Next, we address approximating <f>i{s,y). 



Theorem 4. Under the same hypotheses as the previous theorem, we have 

y) = B{s, y,z) + 0 • 






logP logP 

2-^ p® — 1 ^ B® — 1 

p<2 Z<.p<Z 



Proof. We have 
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Approximating the second term using Lemma 2 , we have 



E 

z<.p<z 



logP 

— 1 





dt 



logy 
- I 



e(y) 



logz 
2^ — 1 



e(^) 




(t(t- - 1) 



st^ ^ log t \ 
- 1)" ) 



dt. 



Making use of the fact that s > 1/2 + <5 and that by the Riemann Hypothesis 
we have e{t) = 0 {'/ilogt), we see that the three error terms are bounded by 
0((logz)Vz‘5). 

To evaluate the integral, we observe that 





dt 



OO 



= E 



1 



1 — fcs 



(y 



1 — ks 



— Z 



1 — ks 



As in the previous theorem, we truncate the infinite sum at fc = [(log y)/sj . 

□ 



And now the theorem for <j)2{s,y): 

Theorem 5 . Under the same hypotheses as the previous two theorems, we have 
Ms, y) = C{s, y,z )+0 ^ ^ 



Proof. We have 



Ms,y) 



^ p^jlogp)^ p^jlogpf 

^ (p^ — 1)^ (p® — 1)^ 

p<z ’ z<p<z ’ 



Approximating the second term using Lemma 2 , we have 
p®(logp)2 



E 

z<,p<z 



y Ulogt , y®(logy)2 2® (log 2)2 

dt + — —e[y) - 7- — 777^(2) 



(p®-l)2 7, (t®-l)2- ' (y®-l)2 

d t®(logt)2 



(2® - 1)2 



-IM 



dt (t® - 1)2 



dt. 



Making use of the fact that s > 1/2 + <5 and that by the Riemann Hypothesis 
we have e(t) = O(Vilogt), we see that the three error terms are bounded by 
0 {{logzf/z^). 
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To evaluate the integral, we integrate by parts to obtain 



r t^ogt zlogz ylogy f^l + logt 

A - 1)2 ^ - 1) s(y« - 1) A - 1) 



Using the expansion 





k=l 



and interchanging summations we then obtain the following for the third term 
above: 



1 

s 







t ^“logtdt 



Integrating, we obtain 




fc=i 



y 



1 — fcs 



1 — fcs 



yi-fe^(logy-l/(l-fcg)) 
1 — fcs 1 — fcs 



‘^'’(loga — 1/(1 — fcs)) 
1 — fcs 



We truncate the sum as before. 



□ 



Finally, our theorem that shows that a/ will approximate a. 

Theorem 6. Under the same hypotheses as the previous three theorems, 
if I log a; — B{s, y,z)\ <C 1, then |s — a| <C l/(log a; log y) . 

Proof. Our proof has two phases: first we show that |a — s| <C l/(logy)2, and 
then we use this to deduce the theorem. 

We begin by observing that (p 2 {s, y),(j) 2 {oi, y) > <(' 2 ( 1 , y) (logy)^. This uses 
the fact that s, a < 1 and that 4>2 is decreasing, and then we estimate the sum 
using the prime number theorem. 

Next, by the mean value theorem there exists a real number t between s and 
a such that 

Ms, y) = M<^, y) + {s- a)Mt, y)- 

By definition, —<f>i{a,y) = log a;; by Theorem 4 we have —(f>i{s,y) = B{s,y,z) 
+ 0 ( 1 ). Combining this information we have 

I I \Ms,y) - M(^,y)\ 

s — a = —7 r 

M't,y) 

\B{s,y, ^) - log a; + 0(1) I 1 

(logy) 2 (logy) 2' 

That completes Step 1. 

For Step 2, we can now assume \a— s| <C 1/ (logy) 2 , and note that we have 
u = u = log a;/ logy as a consequence of our condition that s > 1/2. We take a 
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Taylor series expansion of 4>i about a to obtain 

(/)i(s, y) = y) + Y^ <i^k+ 2 (a, y) 



fc =0 



= y) + (s- a) y) + </*fc+ 2 (a, y) j . 



Focusing on the sum, we use the fact that (j)k{a,y) <C fc!(loga;)^u^“^ (see [7]) 
and substitute our bound for |a — s| to show this sum is bounded by a constant 
times 



(fc + 2) log a; 
^ (logy)'=-i 



0(loga;). 



We now have 



y) = (pi{a, y) + {s- a){(j) 2 {a, y) + O(logx)). 

Using Theorem 4 and the lower bound <() 2 (a, y) ^ (loga;)^/u (see [7]) completes 
the proof. □ 



Corollary 7. Under the same hypotheses as the previous theorems, we have 

HTf{x,y,z,af) = HT{x,y,a) ^ ■ 

Furthermore, Algorithm HT-fast has a running time of 

O (\/y ( 1 / log log y + (log log a;) / log y) ) 
floating point operations. 

Proof. Our results above together with our choice for 2 and the results in [9] 
completes the proof. 



4 Experimental Results 

In this section we conclude with a comparison between algorithms HT and HT- 
fast. 

We implemented both algorithms in C-|— I- and had them compute a list 
of estimated values for 'F{x,y) with x ranging from 2^® up to and y = 

2^®, 2^®, 2^°, 2^^. Due to memory restrictions, it is difficult for algorithm HT to 
handle y values much larger than 2^^. 

For each estimate of 'F{x,y), we also give the values of a, f, 4>i, and (j )2 
computed by each algorithm. We also give the elapsed time in CPU seconds (we 
used a Pentium Pro 200 running Linux kernel 2.2.7). Finally, we give the ratio 
HTf/HT which compares the two estimates of 'F{x,y). 

We used a separate table for each y-value, with the x-values indicated along 
the left. 

Although our theory sets the condition that 5 log x < ,Jy, in practice we used 
the somewhat looser condition log x < ^ with no ill effects. 
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Table 1. Experimental Results: y = 2^® 



X 


Algorithm 


a 




^{x,y) 


Time HTf/HT 


225 


HT 


0.919503 


58.92 -17.33 


113.8 


1.992e-f07 


0.2 






HT-fast 


0.919983 


59.29 -17.38 


114.3 


2.004e-t07 


0.01 


1.0056 


250 


HT 


0.820435 


689.5 -34.66 


256.6 


4.672e-fl3 


0.19 






HT-fast 


0.820947 


699.9 -34.79 


257.8 


4.739e-fl3 


0.01 


1.0143 


275 


HT 


0.767136 


6700 -51.99 


405.6 


3.613e-fl9 


0.2 






HT-fast 


0.767657 


6863 -52.2 


407.5 


3.698e-fl9 


0.01 


1.0234 


2100 


HT 


0.730847 


5.952e-t04 -69.31 


557.9 


1.378e-t25 


0.2 






HT-fast 


0.731373 


6.153e+04 -69.61 


560.6 


1.423e-t25 


0.01 


1.0327 


2250 


HT 


0.621485 


1.36e-tl0 -173.3 


1504 


1.33e-b55 


0.18 






HT-fast 


0.622015 


1.486e-K0 -174.1 


1512 


1.451e-f55 


0.01 


1.0911 



Table 2. Experimental Results: y = 2^® 



X 


Algorithm 


a 


C(a,y) (l^iia,y) 4>2{a,y) 


<I'{x,y) 


Time HTf/HT 


225 


HT 


0.945145 


48.97 -17.33 


124.3 


2.405e-f07 


0.74 






HT-fast 


0.945437 


49.17 -17.36 


124.6 


2.414e-f07 


0.01 


1.0035 


250 


HT 


0.854856 


460.3 -34.66 


282.9 


9.399e-fl3 


0.73 






HT-fast 


0.855158 


464.5 -34.74 


283.7 


9.481e-fl3 


0.01 


1.0088 


275 


HT 


0.806604 


3605 -51.99 


448.9 


1.368e-f20 


0.73 






HT-fast 


0.806906 


3657 -52.12 


450.2 


1.387e-f20 


0.01 


1.0141 


2100 


HT 


0.773847 


2.589e+04 -69.31 


618.7 


1.059e-t26 


0.73 






HT-fast 


0.774149 


2.64e-t04 -69.5 


620.5 


1.079e-t26 


0.01 


1.0194 


2250 


HT 


0.675444 


1.713e+09 -173.3 


1674 


1.68e-b58 


0.65 






HT-fast 


0.675737 


1.802e-t09 -173.8 


1679 


1.767e-f58 


0.02 


1.0514 


2500 


HT 


0.605253 


6.865e-tl6 -346.6 


3491 


9.633e-fl05 


0.65 






HT-fast 


0.605536 


7.591e-tl6 -347.6 


3501 


1.065e-fl06 


0.01 


1.1052 



Table 3. Experimental Results: y = 2^° 



X 


Algorithm 


a 


C(«,y) 


4>i(a,y) </ 2 {a,y) 


T{x,y) 


Time HTf/HT 


225 


HT 


0.964492 


42.35 


-17.33 


134.1 


2.743e-f07 


2.61 






HT-fast 


0.964689 


42.47 


-17.36 


134.4 


2.75e-f07 


0.02 


1.0025 


250 


HT 


0.881284 


333.5 


-34.66 


308.4 


1.581e-fl4 


2.61 






HT-fast 


0.881481 


335.6 


-34.72 


309 


1.591e-fl4 


0.02 


1.006 


275 


HT 


0.837112 


2195 


-51.99 


491.3 


3.745e-f20 


2.62 






HT-fast 


0.837304 


2216 


-52.08 


492.2 


3.781e-f20 


0.03 


1.0094 


2100 


HT 


0.807214 


1.327e-b04 -69.31 


678.6 


5.018e-t26 


2.62 






HT-fast 


0.807403 


1.344e-t04 -69.44 


679.9 


5.081e-t26 


0.03 


1.0127 


2250 


HT 


0.717711 


3.206e+08 -173.3 


1845 


4.277e-f60 


2.62 






HT-fast 


0.717887 


3.309e-t08 -173.6 


1848 


4.414e-f60 


0.03 


1.032 


2500 


HT 


0.654054 


2.514e-bl5 -346.6 


3852 


6.884e-flll 


2.31 






HT-fast 


0.654219 


2.673e-tl5 -347.2 


3859 


7.316e-flll 


0.02 


1.0628 


21000 


HT 


0.592735 


4.912e-t28 -693.1 


7957 


9.998e-f204 


2.62 






HT-fast 


0.592889 


5.516e-t28 -694.4 


7970 


1.123e-f205 


0.03 


1.1228 
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Table 4. Experimental Results: y = 2^^ 



X 


Algorithm 


a 


C(«,y) 4 > 2 { a , y ) 


T{x,y) 


Time HTf/HT 


225 


HT 


0.97945 


37.7 -17.33 


143.5 


3.013e-t07 


9.39 






HT-fast 


0.979551 


37.75 -17.34 


143.6 


3.016e-f07 


0.05 


1.0013 


250 


HT 


0.902068 


256.7 -34.66 


333.2 


2.351e-fl4 


9.56 






HT-fast 


0.902169 


257.5 -34.69 


333.5 


2.358e-tl4 


0.04 


1.003 


275 


HT 


0.861256 


1463 -51.99 


532.7 


8.177e-t20 


9.54 






HT-fast 


0.861354 


1470 -52.04 


533.3 


8.216e-t20 


0.05 


1.0048 


2100 


HT 


0.833713 


7676 -69.31 


737.4 


1.692e-t27 


9.51 






HT-fast 


0.833809 


7726 -69.39 


738.2 


1.703e-t27 


0.05 


1.0065 


2250 


HT 


0.75155 


8.081e-h07 -173.3 


2014 


3.469e-t62 


9.56 






HT-fast 


0.751638 


8.212e-t07 -173.5 


2016 


3.525e-t62 


0.05 


1.0162 


2500 


HT 


0.693294 


1.645e-tl4 -346.6 


4213 


3.274e-tll6 


9.5 






HT-fast 


0.693376 


1.697e-tl4 -346.9 


4217 


3.376e-tll6 


0.05 


1.0312 


2 IOOO 


HT 


0.637262 


2.293e-t26 -693.1 


8712 


1.051e-t216 


9.53 






HT-fast 


0.637338 


2.43e-f26 -693.8 


8720 


1.114e-h216 


0.05 


1.0597 



5 Conclusions 

In this paper we have shown how to drastically speed algorithm HT for com- 
puting estimates of the function y). Our new algorithm appears to be quite 
accurate, and it is much faster than algorithm HT for larger values of y. 
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Abstract. The Elliptic Logarithm Method has been applied with great 
success to the problem of computing all integer solutions of equations 
of degree 3 and 4 defining elliptic curves. We explore the possibility of 
extending this method to include any equation f(u, v) = 0, where / € 
Z[u,u] defines an irreducible curve of genus 1, independent of shape or 
degree of the polynomial /. We give a detailed description of the general 
features of our approach, putting forward along the way some claims 
(one of which conjectural) that are supported by the explicit examples 
added at the end. 



1 Introduction 

Throughout this paper, the term elliptic equation shall mean an equation f{u, v) 
= 0 in rational integers u and v, where / G Z[A, Y] is such that the plane curve 
defined by / = 0 is an irreducible curve of genus 1. The Elliptic Logarithm 
Method — (Sllog for short — as a practical method for solving such equations, was 
first applied by Stroeker and Tzanakis [12] and, independently, by Gebel, Petho 
and Zimmer [6] . Since then, it has been applied extensively to a variety of elliptic 
equations of degree 3 or 4; see [11], [16], [1], [7], [14], [15], [13]. In particular, a 
general treatment of the cubic elliptic equation can be found in [15]. 

Now that many equations have been successfully solved by application of 
(Sllog, it seems natural to ask what we can learn from the experience acquired so 
far, so that we may distinguish the essential characteristics of the method which 
would make its successful application possible to any elliptic equation. We shall 
put forward some plausible suggestions, not all of which we can prove yet in full 
generality. Next we shall test our general observations by a few specific examples 
of non-standard elliptic equations. 



W. Bosma (Ed.): ANTS-IV, LNCS 1838, pp. 551—561, 2000. 
© Springer- Verlag Berlin Heidelberg 2000 
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2 Preliminaries 

Let 

f{u, v) = 0, where / G Z[u, w] is irreducible, 

define an elliptic curve C, birationally equivalent over a number field K of degree 
at most min{deg„ /, deg„ /} to 

S : = q{x) = + Ax + B, 

by means of a birational transformation 

u=U{x,y) , V = V{x,y) 

X = X{u,v) , y = y{u,v) 



(see e.g. [9], Proposition 1). 

Claim 1 One can explicitly calculate a possibly large positive constant M, and 
finitely many parametrizations of C of the form 

u{t)=t-'', u(f) = at'' + + a V' + . . . (1) 

for rational integers v > 1, p < p! < p" < . . ., and non-zero algebraic integers 
a, a' , a" . . such that every real point (u, v) on C with |u| > M can be expressed 
as {u{t), v{f)) by means of one of the parametrizations (1) for a suitable value of 

t. 



Although this claim seems quite classical (Puiseux), the crux lies in the ef- 
fectiveness of the calculation of M. For a proof, see Lemma 5 of [2]. This result 
of Coates, however, is not useful for explicit computations, as it implies an ex- 
tremely large M. Much smaller M is implied by subsequent results of W.M. 
Schmidt [8], and B.M. Dwork and A. van der Poorten [4], [5]. In certain exam- 
ples the numerical values of M generated by these improved results may still be 
very large. For instance, the size of M in our example of section 6.3 is roughly 
10®°; this means we need a method that can detect, in some subtle way, all in- 
tegral solutions {u,v) with |u| < M. At present, no such method is known to 
us. 

Clearly, there is no loss of generality restricting our investigations to those 
solutions {u, v) of f{u, u) = 0 with u > 0. The above claim implies that, for a 
given point (u, v) on the curve and u sufficiently large, the equation /(u, u) = 0 
can be solved for v, i.e. there exist differentiable functions ui(u), . . . , Vk{u) , (fc < 
deg„ /), such that f{u, vfiu)) = 0 identically in u for every i G {1, . . . , fc}. Of 
course, this is also an immediate consequence of the Implicit Function Theorem. 
Let us put 



xoi = lim X{u,Vi{u)). 

u—*oo 

From this point onwards P G C will always denote a point with integral 
coordinates {u{P),v{P)). Since all points P with relatively small coordinates 
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can be easily found explicitly, we may assume u{P) to be sufficiently large, so 
that, for some i G fc}, v{P) = Vi{u{P)) and x{P) = X{u{P),v{P)) is 

close to xoi- Let us explain what we mean by ‘close’ in this context. 

Let Cl denote the only real root of g(x) = 0 if this equation has a single 
root only (the complex case), or the largest real root in case of three real roots 
(the real case); in the latter case the other two real roots are denoted by 62 and 
63 and we assume 63 < 62 < ei. In the complex case, xoi > ei and by ‘close’ 
we mean that x(P) > ci as well. In the real case, xoi G [63,62] and now ‘close’ 
means that x(P) G [03, 62] too. 



3 Two Related Elliptic Integrals 

It is not difficult to see that 

da; du 

— = G(u,v)— -, 

V Jv{u,v) 



( 2 ) 



where 



G(u, v) = 2 



3f„(u, v) ■ fy{u, v) - Jf„(u, v) • /„(u, v) 



3A’^(u, v) + A 

In case /(u, u) = 0 is a Weierstrass equation, a quartic equation of type v'^ = 
Q{u) for some quartic polynomial Q, or a general cubic elliptic equation, the 
function G{u,v) G C(C) is constant; see [12], [16] and [15]. For example, in case 
of a general cubic equation, G{u, v) = ±2. 

Now fix z G {!,..., fc}. For u sufficiently large, y{u,Vi{u)) and X{u,Vi{u)) 
are continuous functions of u; if we denote them by y{u) and x{u) respectively, 
then y(zz)^ = x{u)^ + Ax{u) + B = q{x{u)). Hence y{u) = £^/q(x(u)) with 
£ G {—1, 1}. On putting 

gi(u) = G(u,Vi(u)), 

we have, by (2) and our assumption on the size of u(P), 



gi{u)du 

Ju(P) fv{u,Vi{u)) 

Here x{P) = X{u{P),v{P)) of course. 



L 



da; 



(P) £Vg([r) 



(3) 



4 Necessary Conditions for the Applicability of 



For £hog to work it is essential that the integral in the left-hand side of (3) tends 
to zero as u{P) tends to 00. 

Conjectural Claim 2 



9i{u) 

fy{u,Vi{u)) 






(4) 



for some <5 > 0 . 
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For example, if /(u, w) = 0 happens to be a Weierstrass equation to start with, 
no birational transformation is needed, and 6 = ^, while in case of either a 
non- Weierstrass cubic equation or of a quartic equation of type = Q{u) with 
quartic polynomial Q, it is easily shown that (5 = 1 (see [15] and [16], respec- 
tively) . 

It follows from (4) that the left-hand side of (3) is < ciu~^. Here the constant 
Cl, as well as all other constants Ci in the sequel are effectively computable. 

Claim 3 Let h{-) denote the logarithmic height. Then, 

h{x{P)) = h{X{u{P),v{P))) < C 2 -k calog |u(P)|. (5) 

Inequality (5) is easily seen to be true. Indeed, write 

f{u,v) = fd{u)v'^ + ■ ■ ■ + fi{u)v + fo{u) 

with fj{u) e Z[uj of degree j. If (u,v) € 1? and f{u,v) = 0, then v is an 
integral root of the polynomial fd{u)X'^ -k • • • -k fi{u)X + fo{u) with integer 
coefficients. Hence v divides fo{u), from which it follows that |u| < |/o(u)|. This, 
combined with the fact that X{u,v) is a, rational function of u and v with integer 
coefficients, implies inequality (5). 

We also need the following relation between the Neron-Tate height and the 
logarithmic height (see e.g. [10]): 

h{x{P)) - ih(P) < C 4 . (6) 

Now, the right-hand side of (3) is a so-called linear form in elliptic logarithms 
of points on £(Q), say C{P). It has integer coefficients, which are essentially the 
coefficients of P with respect to a Mordell-Weil basis chosen well in advance, 
and we denote the maximum absolute value of these coefficients by N . A more 
detailed description of C is given in section 5. 

By S. David’s Theorem [3], we obtain a lower bound for C{P) of the shape 

|£(P)| >exp(-c5(log7V + C6)(loglogTV + C7)'=), (7) 

where fc = r-k2orr-k3 and r is the rank of the Mordell-Weil group. We also 
need an upper bound for L{P). This upper bound can be deduced from (3) and 
(4): 

\C{P)\<cr{u{P))-^. 

Combining this with (5), (6) and the well-known fact that h{P) > cgfV^, we 
obtain 

\£{P) \ < exp(-c9fV^ -k cio) (8) 

and finally (7) and (8) imply an upper bound for N. Much of the material found 
in this section and the next consists of straightforward adaptations from [12], 
[16] or [15]. 
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5 The Linear Form C{P) 



In this section we discuss in some detail the linear form C{P), and we show that 
this form indeed qualifies as a suitable linear form in elliptic logarithms of points 
on £1(Q) to which S. David’s theorem, mentioned in the previous section, can be 
applied. 

The curve defined by = q{x), has the identity component £o(®) and 
in the real case — we remind the reader that q{x) = 0 then has three real roots 
Cl > 62 > 63 — also the bounded component £i(M). Let Qj = (cj,0) G £(Q) for 
j = 1, 2, 3. For any R G £i(M) we put R' = R + Q 2 & We have the usual 

isomorphism 

(j) : £o(R) — > [0, 1) = M/Z 

(see e.g. [12]). In the complex case — that is when q{x) = 0 has a single real 
root — i£o(R) = ifl(R) and 4> is defined on the whole of £(M). In the real case 4> is 
extended to a two-to-one epimorphism (p, defined as follows: 

/(/-(i?) ifi?G£o(R), 

’ \(piR') ifi?G£i(M). 



Let oj = 2j 



dt 



■'ei \/W)’ 
convince one that 



the fundamental real period. A bit of thought suffices to 



7, , _ J elliptic log of i? if i? G i£o(R)) 
^ ' y elliptic log of R' if i? G £i(M). 



(9) 



We write 

P = niPi + • • • + TlrPr + T, 

where Pi, ... ,Pr form a Mordell- Weil basis and T is one of the finitely many 
torsion points. It is easy to see that the 4>{T) are rational numbers with effectively 
bounded denominators. Then, 



(p{P) and (f>{—P) are of the form mi(p{Pi) 



r<P{Pr) 



mo 



(10) 



where mj = dzrij (j = 1, . . .,r), mo G Z is effectively bounded in terms of N, 
and s, t are relatively prime integers, effectively bounded by a small number. 

Consider the integral in the right-hand side of (3) and recall that f{u, Vi(u)) = 
0, provided u is sufficiently large. 

Claim 4 



a;oi G QU {±00}. 

The truth of this statement depends only on the truth of Claim 1 as we shall see 
shortly. First note that f{u, v) cannot be a factor of either the numerator or the 
denominator of the rational function X(u,v)). For, otherwise, the whole curve 
C could be mapped into a line, which is impossible for a curve of genus 1. Next, 
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by Claim 1, every point (u, Vi{u)) € C with u near oo has a parametrization (1), 
where the coefficients and the exponents depend solely on the function Vi. On 
substituting the t-expressions for u and v of (1), the value of X{u,Vi{u)) for u 
near oo can be seen to be given by an expression of the form 



+ . . . 

'ftP + j'tP' + '-^"tP" + . . . 



{t near zero), 



where /3, , 7, 7', 7", . ■ ■ Eire non-zero algebraic numbers and A < A' < 

A" < . . . and p < p' < p" < . . . are rational integers. This shows that 



XQi= lim X{u,Vi{u)) 

u—*oo 



Ph if A = p, 
00 if A > p, 
—00 if A < p. 



If xoi yf ±00 we denote by Qoi the point with a;-coordinate xoi and non- 
negative y-coordinate. If xoi = ±00 we set Qoi = the group identity. 

We distinguish two cases: 

I. Cl < xoi- Then, because u(P) is assumed to be sufficiently large, we have 
Cl < x(P) = X(u(P),v(P)) and hence 



da; 



da; 



dP) V^(^) J^(P) 



da; 

v^y 



= oj(p{aP) - up^Qoi) = Lop{aP) - up{Qoi). 



Here ct = 1 or —1, depending on whether y{P) = y{u{P),v{P)) is non- 
negative or negative, respectively. This, combined with (10) and (9) shows 
that the integral in the right-hand side of (3) is equal to a linear form in 
elliptic logarithms 



— Uj4>{Qoi) + (too -I- -)w -I- miU!(j){Pi) -I- • • • -I- mrUJ(f){Pr), (11) 

and all points appearing in it have algebraic coordinates. 

2. xoi G [ 63 , 62 ]. Then, because u{P) is sufficiently large, x{P) G ( 63 , 62 ) and 



da; 



da; 



<p) J^(p) 



da; 



da; 



da; 



i/gOry Jx(P') vWy 



= u!(j){aP') - uJpiQoi) = ujp{aP) - w^(Qoi) 



and we arrive at the same conclusion (11) as before. 



6 Examples 

It is not easy to find in the literature non-trivial examples of irreducible curves 
of genus 1 of an unusual shape, that is given by equations of degree at least 
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5. Therefore, with the exception of the third example, we have generated a few 
examples by ourselves. Further, we shall only discuss solutions (u, v) with u > 0 
and sufficiently large. 

We have chosen not to take (Sllog ‘all the way’, for the simple reason that, 
once we have checked the various claims — and this is what we actually do 
below, except for the values of the various M’s^ — completing the computations 
is merely a routine matter, be it a tedious one. 

We have implemented in Maple a procedure for computing parametrizations 
(1), using Newton polygons (see e.g. [17]). 



6.1 Three Simple Examples 

We have grouped the following three equations because of their similarity; each 
provides a straightforward example of an elliptic equation of unusual form. In 
the table below we have gathered the relevant information. 



Three simple elliptic equations f(u, v) — 0 



f(u,v) 


M® + 


M® + - 2V^ 


+ — 2v^ 


Singular points 
[m, ii] (multiplicity) 
Rank r 

Weierstrass A, B 


[0,0](3),^(2) 

0 

0,1 


[0,0](2),«d(4) 

1 

0,8 


[0,0](2),«)(5) 

1 

0,8 


Birational transformation 






X{u, v) 
y(u,v) 


O V 


rj 2u^ +14^ 


rj 4-4i; 


^u{u+l) 
u — 1 


^ — 1)^ 


^ u‘^(u-iy^ 

A ~\-3uv-{~5v 


li+1 


— 1)^ 


u'^ (li — 1)^ 


Claim 1 








V 

, . . . 

a, oc\ ol" ^ . . . 

P 

k 


3 

-5, -2, 1,4, 7,... 

p p 5p lOp 

Cl 3 1 9 1 81 1 243 1 • • • 

1/^ 

1 


1 

-3, 0,3, 6, 9,... 

„ P P P 5p 

Cl 2 1 8 1 16 1 128 1 • • • 

±1/^2 

2 


2 

-7,-1,5,11,17,... 

„ P P P Sp 

Cl 2 1 8 1 16 1 128 1 • • • 

±1/^2 

2 


Conjectural Clain 
<5 


1 2 

1/3 


1 


1/2 


Claim 4 








Xio{u oo) 


0 


4 ±4^2 


2 



^ We actually believe that the various series Vi(t) do converge for |t| less than some 
number of the order 0.1 say, but we cannot prove this. 
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6.2 A Parametric Family of Degree 5 Curves 

In the course of constructing suitable examples, we struck on the following para- 
metric family of elliptic equations: 

/(u, v) = v^{v — u — l)(u -I- (2r — l)u — 1) -k tu^{v^ — 1) = 0. (12) 



For each value of the parameter r yf 0, r G Z, this equation represents an 
elliptic curve Cr- The singular points of Cr are (u, v) = (0, 0) and (0, 1), both of 
multiplicity 2, and the point at infinity is a singular point of multiplicity 3. The 
birational equivalent curve £r is 



+ ArX + Br, with Ar = — and Br = -k r®, 

and the corresponding birational transformations are (one way only) given by 

, 1 2 -AV \ Tv{-1 + TU - U + v) 

X{u,v) = -T^ - TV, y{u,v) = . 

3 u 

In this example k = 2, i.e. there exist two parametrizations near u = oo. The 
first parametrization is given by^ 



ui{t) = t \ 



v\{t) = —Tt — 2(r — l)t ^ H k2 



1 1)^* (4r-5)(r- 1)^ 



-t- 



, ^ (4r - 7)(r - 1)^3 16r® - 104r4 -k 259r® - 310r2 -k 182r - 42^4 

+2 4 t = t 

, ^ (r - l)(16r® - 120 t 4 -k 333r® - 430r2 -k 270r - 66 ) ^5 

“k2 - t 



64r^ - 688r® -k 2928r® - 6495r^ -k 8288r® - 6174r2 -k 2508r - 429 

+ 0{t^) (t^O). 

It is obvious from this that 

a;io = lim X{u,vi{u)) = 00 . 

u—*oo 

For this parametrization we find 






5i(u) 



fy{u,vi{u)) 



2-2 ^'''“1-3 „4r — 9r-k6 _4 

= -u ^-4 — —u ®-k2 r u ^ 



8 6r® — 16r^ -k 17r — 7 _r _g 
u ® + 0(u ®)(u^oo), 



^ Although not really necessary, we calculated quite a number of terms in order to see 
what they are like and to demonstrate Maple’s capabilities. 
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SO that (5 = 1 in this case. 

The second parametrization is 



U2{t) = t \ 
V2{t) = Pr- 



2(r-l)(4r2p2 



lOrp^ + + ITt'^Pt + 2pr — 8rpr — 6r + 3r^) 

59r^ — 48r^ + 24r — 4 



”*"r(59r^ 



____^(_80p2 + 864rp2 + 10328r3p2 + I6574r^pl 

-ITOOOrV - 3904r^p2 _ gTllrV? + 1588r^p^ 
+2088r^p^ - 3458rV + 6074r®p^ + 1192r3p^ 

-5270rVr - 208r^p^ + 16rp^ + 1132r^ + 80r 
+7588r® + 2808 t 3 - 4695r® - 752r^ - 6192r^)t2 
+ 0{t^) {t 0 ), 



where pr satisfies the cubic equation + (l/r — 2)X^ — 1 = 0. For this 
parametrization we find 

X20 = lim X{u,V2{u)) = - PrT. 

u—*oo O 

and 

f .. = d 2 U~'^ + 0{u~^) {u oo), 

fv[U,V2(U)) 

where c ?2 can be (and was) explicitly calculated by Maple, but is too complicated 
to be included here. Because ^2 0, <5 = 1 for this parametrization as well. 



6.3 An Example Taken from Maple’s Help Facility 

The Help Topic of the Maple V Release 5.1 command algcurves [singularities] 
makes use of the following curve of rank 5: 

/(u, v) = 180u® - 207u^u - 8u® - 450u^ + 621^3^ - I28uv^ - 35u^ + 

369u^ - 521u\ + 82r;3 - lOOu^ + 135uu - - 7u - 28u + 8 = 0. 

Singular points (all of multiplicity 2) are (u,v) = (0, 1), (1, 0), (1, — 1) and the 
two complex points (u,v) = A short Weierstrass model of this 

curve is 

2 ^3 62058288278602561^ , 61852994116858326481398145 

^ 805306368 ^ 59373627899904 ' 

The corresponding birational transformations are given by 
, . 43681 NumA(u, ti) 

“ 49152u(u2 + l)(u- 1)2’ 

^ 9129329 NumJ^(u, u) 

“ 524288u(u2 + l)(u- 1)3’ 
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with 

NumA'(u, v) = 103981U® + 15228u^t> + 10284uV + 1536u^v^ 

+4128uw^ - 316526u^ + 47412u3w + 67584^2^2 + l5468ut>3 
-2592w^ + 368606u^ - 71388u\ - 88968uw^ - 13932t>3 
-206150u^ + 2268UW + 12636w^ + 52681u + 6480w - 2592, 
Numy(u, v) = 2070033U® + 70533u®u - 28045u'‘t>^ 

+45962^3^3 + 90616^2^3 - 7973144^® + 1130670^3^ 
+1634455u3t,2 + 312517u^t>3 - 117296ui;3 + 12052790^3 
-2569492^3^ - 3224660uV + 524456ut>3 + 33368t>3 
-9090868^3 + 1336366u^w + 1787607uw^ + 179353t>3 
+3599145u^ + 115343UW - 162669w^ - 691324u - 83420w 
+33368. 



In this example there exists only one parametrization near u = oo, given by 



ui{t) = t \ 

vi{t) = pt~^ + do{p) + di{p)t + d 2 {p)t^ + Gif’) (t 0) 



with 



do{p) 

di{p) 

d2{p) 



117652915 4 , 59690773 3 , 64881275 2 _ 37533284 , 3292350 

2647875132^ “'" 294208348" “'“ 294208348" 73552087" “'“ 73552087’ 

2409249577008465 4 _ 143100375932054279 3 _ 3841218563243545585 2 

86558552032889104^ 4154810497578676992^ 12464431492736030976" 

442118719850886867 , 99742932488150451 

692468416263112832" “'“ 173117104065778208’ 

_ 46304367990791457732640885 4 , 91871979044861844697522343 3 

3667139798237041673525787648" “'“ 1833569899118520836762893824^ 



I 43666801880702130891932691 2 , 2831900188941035651896208357 

“'“814919955163787038561286144^ “'“ 29337118385896333388206301184^ 



213000092757640705570148071 

814919955163787038561286144’ 



where p is the only real root of + 207X — 180 = 0. Standard, but tedious 
computations yield 



a;io= lim dflu, t>i(u)) = |^(4128p^ + 1536p3 + 1028V + I5228p + 103981) 

u— »-oo 



and finally 



9i{u) 

fy{u,vi{u)) 



3208960 4 _ 

19764496521^ 

7380608 1 -2 

2196055169/ “ 



3488000 3 I 3609248 2 , 18542144 

19764496521^ “'“ 6588165507/^ “'“ 6588165507/^ 

+ 0{u~^) {u oo). 



which in particular implies that (5=1. 
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Abstract. For integers a and b we define the Shanks chain pi,p 2 , ■ ■ ■ ,Pk 
of length fc to be a sequence of k primes such that pi+i — api^ — b for 
i = 1,2, . . . , k — 1. While for Cunningham chains it is conjectured that 
arbitrarily long chains exist, this is, in general, not true for Shanks chains. 
In fact, with s = ab we show that for all but 56 values of s < 1000 
any corresponding Shanks chain must have bounded length. For this, 
we study certain properties of functional digraphs of quadratic functions 
over prime fields, both in theory and practice. We give efficient algorithms 
to investigate these properties and present a selection of our experimental 
results. 



1 Introduction 

Let e G {+1,-1} be fixed. A Cunningham chain pi,p 2 ,ps, . . . ,Pn of length k 
(see Guy [9], §A7) is a sequence of k primes such that 

Pi+i = 2pi + e (z= l,2,...,fc- 1) . 

For example, if e = 1, we say that 

2,5,11,23,47 

is a Cunningham chain of length 5. The longest known chains of Cunningham 
primes have recently been determined by Forbes [8] . For e = 1 the longest chains 
have fc = 14 (one of these has pi = 23305436881717757909), and for e = — 1, 
the longest known chain has fc = 16 (pi = 3203000719597029781). Indeed by 
Schinzel’s [16] [17] Conjecture H one would expect for either value of e and any 
given fc > 1 the existence of an infinitude of Cunningham chains of length fc. 
Consider now the quantitative version of Conjecture H, given by Bateman and 
Horn [4]. 

* Research supported by NSERC of Canada grant #A7649 



W. Bosma (Ed.): ANTS-IV, LNCS 1838, pp. 563—580, 2000. 
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Hypothesis H. Suppose fi, f 2 , ■ ■ ■ , fk are polynomials in one variable with all 
coefficients integral and leading coefficients positive, their degrees being hi, h 2 , ■ ■ ■ 
hk respectively. Suppose each of these polynomials is irreducible over the field of 
rational numbers and no two of them are identical. Let P{N) denote the number 
of positive integers n between 1 and N inclusive such that fi{n), f 2 {n), . . . , fk{n) 
are all (positive) primes. Then as N ^ +oo we have 



P{N) 



C{fi,f 2 ,...,fk) du ( ['' du \ 

hih 2 ---hk J 2 {logufi ^yj 2 (^ogufi J 



where 



c'(/i,/2,...,/fc)=n 




-k 



1 - 



MY 



the product being extended over all primes and uj{q) being the number of solutions 
of the congruence 



fi{x)h{x)--- fk{x) = Q (mod g) . (1.1) 

For the case of Cunningham chains of length k we have 

fi{x) = X, f 2 {x) =2x + e, fsffi) =4x + 3e, . ..,fk{x) = + (2'"“^ - l)e . 

For any given odd prime q, let v{q) denote the multiplicative order of 2 modulo 
q. We must have v{q) \ q — 1 and 



uj{q) 



k when k < i>{q) 
v{q) otherwise . 



Thus, u{q) < q — I < q and C{f\, f 2 , . . ., fk) is positive. We can therefore assert 
under this conjecture that if P{N) is the number of Cunningham chains of length 
k starting with some pi < N, then as IV ^ 00 , we have 



P{N)^C{fi,f2,... 




Evidently, P{N) goes to infinity as N does. 

In 1963 Shanks [19] observed (under Hardy and Littlewood’s [10] Conjecture 
F) that if 

Q{n) = fi={x : 0 < X < n, — 17 is prime } , 



then 



Q{n) ~ 1.1803 



dx 
log a; 



Since the value of the constant 1.1803 exceeds 1, this caused him to write the 
following passage in a letter [18] to D.H. and Emma Lehmer in 1969. 
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“n^ — 17 has a higher prime density than n itself, even though it grows 
twice as fast.... 

It follows that prime chains 

Pi+i = (2pif - 17 

should be [a] little longer than 



Pi+i =2pi+l 



even though they grow twice as fast. I never did run it though. Try it 
sometime.” 

There is no record of any response to this by the Lehmers, possibly because 
the doubly exponential growth rate for the pi values rapidly produces numbers 
that certainly would have been too difficult to test for primality by the methods 
available at the time. In a short computer trial we discovered that if pi = 3, 
then Pi,P 2 ,P 3 ,Pa are all primes and if p\ = 303593, then Pi,P2,P3,P4,P5 are all 
primes. We were unable to find a sequence of 6 primes for any pi < 6200000. 

For a given pair of integers a, b, we set f{x) = ax^ — b and define fi{x) = x, 
fi^i{x) = f{fi{x)) {i = 1,2, . . .). We define the corresponding Shanks chain of 
length fc to be a set of primes 



Pl,P 2 ,P 3 , ■■■,Pk 

such that Pi = fi{pi) {i = 1, 2, . . . , fc). It seems that Shanks believed that when 
0 = 4 and 6 = 17 one might be able to get somewhat longer chains of primes 
(starting with pi < N for a given N) than the Cunningham chains. However, 
this is not the case. For consider the prime 59. It turns out that for any integer 
X one finds that 

fi{x) = 0 (mod 59) 

for some i < 17. Thus, the maximum chain length possible for this Shanks chain 
is 16. That is, if fc > 17, then the number of solutions of (1.1) is 59 when q = 59 
or w(59) = 59. Hence, C{fi, / 2 , ■ ■ ■ , /fc) = 0 if fc > 17. The question that now 
comes to mind is: how often does this phenomenon occur? To investigate this 
question we first note that 



afi+i{x) = (afi{x))'^ - ab . 

Thus, for an integer s we will define g{x, s) = x'^ — s, go(x, s) = x, gi+i(x, s) = 
g(gi(x, s)) (i = 0, 1, 2, . . .). It follows that if s = ab, we get 

fi{x)=gi-i{ax)/a, (1.2) 

and if q does not divide a, then (1.1) has just as many solutions as 

fc-i 

n 9i{x, s) EE 0 

i=0 



(mod q) . 



(1.3) 
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We now turn our attention to the problem of determining whether for a fixed 
integer s there exists some prime q and some minimal k(> 0) = k{s) such that 
q does not divide s and for any integer x 

gi{x,s) = 0 (mod g) (1.4) 

for some i < k. That is, when is to{q) = q for fi{x) given by (1.2) and k > k7 
We have already seen that if s = 9 (= 68 = 4 • 17 (mod 59)), then this must 
be the case for p = 59 and k = 16. In Table 1 we present for values of q < 200 
those values of s (mod q) and corresponding values of k such that (1.4) must 
hold for some i < k. Note that for these values of s the maximum possible chain 
length of a Shanks chains is k{s). 



Table 1. 



Q 


s, k{s) 


9 


s, k{s) 


Q 


s, k{s) 


2 




59 


9,17; 25,11 


137 


87,27; 118,31 


3 


1,2 


61 


39,15; 45,11; 48,9 


139 


41,22; 107,21 


5 


4,3 


67 


62,16 


149 


96,32; 129,24 


7 


1,3; 4,3 


71 




151 


94,35; 127,20; 137,27 


11 


5,5 


73 


61,22; 


157 


16,33; 100,32 


13 


9,6; 10,5 


79 


8,17 


163 


84,20; 135,29 


17 




83 


37,21; 51,21 


167 


49,26 


19 


9,7; 17,7 


89 


10,18 


173 


151,40 


23 


1,9 


97 


54,18; 66,25 


179 




29 


5,8; 22,10; 25,11 


101 


54,25 


181 


136,25 


31 


16,8; 19,10 


103 


18,20 


191 


64,24; 104,24 


37 


33,13 


107 


102,22 


193 


4,39; 126,25 


41 


25,15; 39,13; 40,9 


109 


38,17; 82,27 


197 




43 


17,8 


113 




199 


47,26; 98,32; 103,23 


47 




127 


35,20; 74,30; 87,17 






53 


44,14 


131 









A glance at Table 1 reveals that for many values of s we would not expect 
to have arbitrarily long Shanks chains. The purpose of this paper is to examine 
when uj{q) = q for a particular value of s. 

2 Generators 

Clearly, we are dealing here with a problem which involves iterating a nonlin- 
ear function modulo a prime; such problems, even for functions as simple as 
quadratic polynomials, are notoriously difficult. However, it is well known (and 
easy to prove) that for any integer x, there exist a least integer m and a least 
integer n > m such that 



gm{x, s) = gn{x, s) 



(mod q) . 





A Note on Shanks’s Chains of Primes 



567 



We denote this value of m by A = A(x, s) and the value of n — m by /i = /i(x, s). 
A is called the tail length, ^ is called the cycle length and p = + X = n 

is called the /9-length of x with respect to g{x, s) and q. Note that the values 
of gi(x,s) (i = 0, 1,2, . . . , p — 1) are distinct modulo q, but that gp{x,s) = 
g\{x,s) (mod q). Many probabilistic results are known concerning p, X, p for 
the iteration of random function (see Flajolet and Odlyzko [7] for several such 
results and references), and in [7] it is postulated that the properties of quadratic 
functions modulo an integer should be asymptotically the same as those of the 
class of all functions. Indeed, Bach [1] has proved that in initial stages, at least, 
quadratic functions do behave asymptotically like random functions. Thus, the 
expected values of p, X, p here should have values close to y/iiq/S, y^irq/S and 
\JtuiI2, respectively. Furthermore, the expected maximum values of p, X, p are 
respectively asymptotic to c\-,/q, C 2 ^/q, c^^/q, where ci « .78248, C2 « 1.73746, 
C3 « 2.4149. (See [7]). 

Now consider the functional digraph of g{x, s) over F^, the finite field of q 
elements. This is the directed graph whose nodes are the elements of and 
whose edges are the ordered pairs (x,g{x,s)) for all a; G F^. For example, the 
functional digraph of g{x, 3) over Fn is 



7 




3 

Each connected component of the functional digraph contains exactly one cycle. 
Thus, in the case of g{x, 9) over F59 the functional digraph has exactly one 
component with its cycle containing the node 0. 

We say that s is a generator for a prime q if for any integer x, there exists 
some minimal z (> 0) such that 



gi{x,s) = 0 (2.5) 

in Fq . Thus, s is a generator for a prime q if and only if the functional digraph of 
g{x, s) over F^ has a single connected component whose cycle contains the node 
0. If s is a generator, we define k{= k{s)) = max{z} of all the values of z given 
by (2.5). Thus, if s is a generator, then any a; G F^ can be written as 



X = ± 




± 





yrr: 



with no more than k radicals. Since the expected number of connected compo- 
nents for the functional digraph (under the same caveats as those mentioned 
above) is (logg)/2 (see [7]), we would not expect to find many generators for a 
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given q and this is borne out by computations (see §4). Evidently, if s = a& is a 
generator for some q, then uj{q) = q for f{x) = ax^ — b and k > k{s); the length 
of any corresponding Shanks chain can, therefore, not exceed k{s). 

We now develop a technique for determining when s is a generator for a 
given q. Of course, this seems to be a very simple task because all we need do 
is start at some node no G F, and by iterating g at this node compute the set 
Sno of all distinct nodes in its tail and cycle C„q over F^. If Sng = F^, we are 
finished, but if Sng F^, we select ni ^ and repeat the process. If for some 
h we get Cno = Cni = • • • = Cn^ and = F^, then s must be a generator; 

otherwise s is not. The difficulty with this very simple algorithm is that when it is 
implemented on a computer and q is large, the amount of memory management 
required during its run greatly degrades its performance. 

We can develop an algorithm for proving that s is a generator for a given q 
which does not involve a great deal of memory management if we are willing to 
do some extra work. To this end we define the following subsets of F^. We put 
TZo = {0} and define TZi+i recursively from TZi by 

TZi+i = {t : = r + s, r G TZi, t yf 0} . 

For example, ifq = 367 and s = 1, we have TZq = {0}, TZi = {±1}, 7^2 = {±288}, 
7^3 = {±17}, 7^4 = {±237}, TZo = 0 - 

We next establish some very simple results concerning TZi. 

Lemma 2.1. If g{x, s) G TZi and x then x G TZi+\. 

Proof. We have x'^ = g{x, s) + s. Since a; yf 0, we must have x G TZi+\ because 
g{x, s) G TZi. 

Corollary. If gj{x, s) = 0 and gj-i{x, s) yf 0 (0 < z < j), then x G TZi. 

Proof. Follows easily by induction on j. 



Lemma 2.2. If x G TZj (j > 0), then g{x, s) G TZj-i. 

Corollary. If x G TZj (j > 0), then gi{x, s) G TZj-i (0 < z < j). 

Theorem 2.3. If i > 0, then 

TZi = {x : gi{x,s) = 0,gj{x,s) yf 0 (j = 0, 1, 2, . . . , z - 1)} . 
Proof. Follows easily from the corollaries of Lemmas 2.1 and 2.2. 
Corollary 2.3.1. If j > i, then TZi H TZj = 0. 

Proof. If a; G TZj, then gi{x,s) ^ 0 (z < j), which means that x ^ TZi. 
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Corollary 2.3.2. If 

k 

Y,ifn, = q, (2.6) 

i=0 

then s is a generator for q. Conversely, if s is a generator for q, then (2.6) holds. 

Corollary 2.3.2 can be used by a computer to prove that s is a generator for 
q, and, as we can produce TZi+i from TZi only and the values of fflZi tend to 
be small, the memory requirements are modest. Of course, we must compute 
roughly ffIZi /2 square roots modulo q to produce TZi+i, but in practice, the 
Tonelli-Shanks algorithm (see Bach and Shallit [2], pp. 155-157) for doing this is 
very efficient; moreover, we developed a method based on the continued fraction 
expansion to compute square roots modulo q that is by roughly a factor of two 
faster than the Tonelli-Shanks algorithm (see §4). That the values of fflZi tend 
to be relatively small follows on noting that in the case that (2.6) holds k must 
exceed the maximum A value and is likely close to the maximum p value. Since 
we expect that about half of the values of r in TZi are such that ((r -|- s)/q) = 1, 
we expect that fflZi+i « fflZi. Thus, kfflZi « g or fflZi « q/k which will likely 
be less than ^/q!c^. However, it turns out that in practice the average value of 
fflZi tends to be much smaller than ^/qjc^. 

We can produce more useful necessary conditions for s to be a generator for 

q- 

Theorem 2.4. If s is a generator for q and q > 2^ — 1, then T^i, 7^2, . . . ,IZj ^ 

Proof. Since the degree of gi{x,s) as a polynomial in x is 2*, there can be at 
most X)i=o 2* = 27 — 1 values of x in such that gi{x,s) = 0 for z < j — 1. Since 
2 ^ — 1 < q, there must (if s is a generator for q) be some x such that gk{x, s) = 0 
{k > j — 1) and gi{s, a;) yf 0 for all 0 < z < fc — 1. Since x € IZk, we have IZk yf 0 
and therefore T^i, 7^2, 7^3 , . . . ,IZj ^ 

From this result we readily conclude that 1 cannot be a generator for 367 
because 367 > 2® — 1 and 77.5 = 0 - Furthermore, if {s/q) = —1, then 77i = 0 
which means that s cannot be a generator for q. Note that this theorem provides 
a possible technique for eliminating a given s as a possible generator for q. If 
q > 2 ^ — 1 for some conveniently selected value of j, we need only compute the 
sets 77i, 77-2, 77-3 , . . . ,IZj. If we find an empty one, then we know that s cannot 
be a generator for q. If we consider the special case of s = 1, we know that 
77i = {±1} and 772 = {a; : x'^ = 2 (mod q)}. We get 772 = 0 if (2/?) = —1. 

If (2/q) = 1, then IZ 3 = {x : x"^ = l-|-t, t^ =2 (mod q)}. Now (1— t)(l-|-t) = 
1 — = —1; hence, if g = 1 (mod 8), then {{l — t)/q) = {{l + t)/q). Thus 

773 = 0 if ((l+t)/g) = -l. 

By a result of Barrucand and Cohn [3] , we know this can only occur when q 
cannot be represented by the quadratic partition a^-|-326^. We have the following 
theorem. 

Theorem 2.5. 1 cannot he a generator for any prime q such that q = 3,5 
(mod 8) or for any prime q = 1 (mod 8) such that q^ of + 32b^. 
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3 Cycles 

We mentioned earlier that it seems an unlikely event that any particular s will 
be a generator for a given q. Certainly, s will not be a generator if the functional 
digraph of g{x, s) over contains a cycle which does not have 0 as a node. In 
this section we will investigate the problem of the existence of small cycles. We 
will consider the cases of minimal /i = 1, 2, 3, 4 only. For g, = 1, we want to know 
whether there exists some x such that g{x, s) = a; in F^. Clearly, this will be the 
case if and only if ((4s + l)/( 7 ) = 0, 1. Furthermore, this cycle is made up of the 
node X only when {2x + 1)^ = 4s + 1 (mod q). Since a; ^ 0 (mod q) under 
this condition, we have proved the following simple result. 

Theorem 3.1. If ((4s + l)/q) = 0, 1, then s is not a generator for q. 

We also have the following result. 

Theorem 3.2. If s = t^ — t 0), then s ean never he a generator for any 
prime q. 

For the case that /i = 2, we must have g 2 {x, s) = x and gi{x, s) yf a; in F^. 
Now it is easy to see that 

52 ( 3 ;, s) - a; = ( 5 i(a;, s) - a;)(a;^ + a; - s + 1) ; 

thus, we have a cycle of length 2 if and only if ((4s — 3)/q) = 0, 1. However, 
since this cycle is made up of the nodes {a;, —1 — a;} when (2a; + 1)^ = 4s — 3 
(mod q), we see that 0 is not in the cycle only when s ^ 1 (mod q). 

Theorem 3.3. If s ^ 1 (mod g) and {{4:S — 3)/q) = 0,1, then s is not a 
generator for q. 

Also, if s = 1 + 1, then s is not a generator for any q unless s = 1 (mod q) 

and 1 is a generator for q. 

Theorem 3.4. If s = t‘^+t+1, (f < B), then s can never he a generator for any 
prime q if t ^ 0, —1 (mod r) for all primes r < B such that 1 is a generator 
for r. 

Now 1 is a generator for r = 3,7, 23 and for no other prime < 5000. Thus, 
if s = t + 1 (t < 5000) and t ^ 0,-1 (mod 3), t ^ 0,-1 (mod 7) and 
t ^ 0, —1 (mod 23), then s can never be a generator for any prime q. 

Notice that for the forms of s given by Theorems 3.2 and 3.4 we can never 
have uj{q) = q when s = ah. Thus, we should (under Conjecture H) be able to 
find corresponding Shanks chains of arbitrary length. Also for these forms we 
have solutions in Z of gi{x,s) = x. However, by a result of Narkiewicz ([14], 
Theorem 2), we know that there are no values of a; in Z such that the least cycle 
length of g{x, s) exceeds 2. This suggests (but certainly does not prove) that the 
only possible values of s which can never be a generator for any q are those given 
in Theorems 3.2 and 3.4. 
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We next consider the case of cycles of length 3 over F^. This has been ex- 
amined for arbitrary fields of odd characteristic by Morton [12], but we will use 
a somewhat different and computationally more convenient approach here. We 
have 

5 s(a;, s) - X = {gi{x, s) - x)c(x) , 

where c(x) = 

x^ + x’^ + (l-3s)x^ + (l-2s)x^ + (l-3s+2s^)x^ + (l-2s+s^)x + l-s+2s^-s^ . 

If we have a cycle of length 3 for g(x, s) over F^, we must have three distinct 
zeros of c(x), because if x is a zero of c(x) in F^, then so also must be 51 (x, s) 
and g 2 (x, s). That is, x^ — s and (x — s)^ — s must be zeros if x is. This follows 
from the simple observation that gi+j{x, s) = gi{gj{x, s),s). By an old result of 
Escott [ 6 ], we know that if c(x) is to have a zero in F^, then s = a?' — a + 2 for 
some a e Fq. When this happens, we get 

c(x) = (x^-|-ax^-|-(a— s— l)x— (as — s-|-l))(x^-|-a^x^-|-(a^ — s— l)x — (a^s— s-l- 1 )) , 

where a' = 1 — a. Then, a necessary condition that there be a cycle of length 3 
for g(x, s) over F^ is that ((4s — l)/q) = 0, 1. Now if g yf 3 and s = — a + 2, 

we see that 

p{x) = x^ -I- ax^ + {a — s — l)x — (as — s -I- 1) 
will have 3 zeros in F^ if and only if 

h{x) = x^ -I- 3(2a — 1 — 4s)x — (16as -I- 2a — 20s -I- 13) 
has 3 zeros in F^. Since 

16as -I- 2a — 20s -I- 13 = (4a — 3)(4s — 2a -I- 1) , 
it is a simple matter to evaluate the discriminant D of h{x) as 

D = -27(4s- 2a-k 1)^ . 

If 4s — 2a -I- 1 = 0 in Fq, then p(x) has the zero —a/3 with multiplicity 3; 
furthermore, if x = —a/3, then g(x,s) = x. Thus, we exclude this possibility 
and we find that {D/q) = {—3/q) = q (mod 3). By classical results concerning 
the solubility of cubic congruences modulo q (see Dickson [5], p. 256) we know 
that h{x) can have 3 zeros in F^ if and only if q = {D/q) (mod 3) and 

a(«'-i)/3 = 1 (mod q) , 

where a = {A+\/D)/2, A = 16as-|-2a— 20s-|-13. We note that a = (4s— 2 a-|-l) 7 , 
where 7 is a zero of x^ — (4a — 3) x -I- 4s — 2a -I- 1. If C is a primitive cube root of 
unity (C^ + C + 1 = 0)) then 7 = 2 a -I- 3^. 

If (7 = — 1 (mod 3), then -i)/3 = [\/q] (mod q), where [\/q] is de- 
fined to be the value of C* in Z[C] such that = (j- (mod q). li q = 1 
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(mod 3), then = [o/tt], where tt is a primary prime factor of q in Z[C] 

and [a/ir] is that value of Q such that -i)/3 = Q (mod tt). Thus, if q does 

not divide 4s — 2a + 1 and q ^ 3, then p{x) has three zeros in if and only 
if [ 7 /( 7 ] = 1 when q = —1 (mod 3) or [(4s — 2a + 1 ) 7 / 71 ] = 1 when q = I 
(mod 3) and tt is a primary prime factor of q. Also, if a cycle of length 3 exists 
for this s and a and as — s + 1 7 ^ 0 in F^, then none of the zeros of p{x) can be 
zero. We have proved the following theorem. 

Theorem 3.5. If q is a prime, q ^ 3, q does not divide 4s — 2a + 1 and 

((4s — 7)/q) = 0, 1, then there is a cycle of length 3 for g{x, s) over F^ if and 

only if s = of — a + 2 (mod q), 7 = 2 a + 3C and [ 7 / 9 ] = 1 when q = —1 
(mod 3) or [(4s — 2a + 1 ) 7 / 71 ] = 1 when q = 1 (mod 3) and it is a primary 
prime divisor of q. Furthermore, if a cycle of length 3 exists for this value of s 
and a, and q does not divide as — s + 1 , then s cannot he a generator for q. 

We remark here that the values of [^/t/] here can be obtained quite rapidly by 
making use of the idea of Jacobi (see Williams and Holte [21]). 

If we consider the case of s = 1, we see that we must have {—3/q) = 1 in 
order to have p = 3. Thus, we require that q = 1 (mod 3) and we may put 

4q = L‘^ + 27M^ (L = 1 (mod 3)) 

as an essentially unique (up to the sign of M) quadratic partition of 4q. Since 
of — a + 1 = 0 (mod q) is soluble for a & "L, may put q = where 

7 Ti I a + C, 7 T 2 I a — 1 — C and tti, 7 T 2 are primary prime divisors of q in Z[Q. We 
have 

(4s - 2 a + 1)7 = (5 - 2 a)( 2 a + 3C) = C(5 + 2 C) = 3C - 2 (mod tti) . 

Also, 4s — 2a + 1 = 5 — 2a yf 0 if (7 yf 19. Since 2 — 3^ is a primary prime divisor 
of 19, we get 

7 ( 4 s- 2 a+l )1 _ r3C-2l _ r2-3C] _ r tti 
7 Tl J L ’’’1 J L ’’’1 J [2 ~ 3 C 

by the law of cubic reciprocity. Now we select the sign of M such that tti = 
{L + 3M)/2 + 3Mf and note that C = —12 (mod 2 — 3C); hence, tti = (T + 
3M)/2 — 36M (mod 2 — 3C.). Thus, [ 7 Ti /(2 — 3p)] = 1 if and only if 

((L + 3M ) /2 - 36M)® = 1 (mod 19) . 

Since 2^® = 8 ® = 1 (mod 19), this is equivalent to (4L + 9M)® = 1 (mod 19) 
or 4L + 9M G {±1,±7, ± 8 } (mod 19). We now have the following result for 
s = 1 . 

Theorem 3.6. // s = 1 and (7 yf 3, 19, there is a cycle of length 3 for g{x, s) 
over Fq if and only if q = 1 (mod 3), 4q = + 27M^ and either AL + 9M 

or AL -9M G {±1,±7,±8} (mod 19). If both AL + 9M and AL - 9M G 
{±1,±7, ± 8 } (mod 19), then there are exactly two cycles of length 3. Also, 
1 cannot he a generator for any q such that g{x, s) over F^ has a cycle of length 
3. 
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If we consider the example of q = 157, we get L = —14, M = ±4. Since 4 • 
(—14) + 9 • 4 = —1 and 4 • (—14) — 9 • 4 = 3 mod 19, we see that there is a 
single cycle for s = 1; this is {92, 142, 67}. If q = 151, we get L = 19, M = ±3 
and 4L ± 9M = ±8 (mod 19). Thus, in this case we get two cycles, namely 
{19,58,41} and {85,127,122}. 

For the case of /i = 4, it is convenient to use the techniques of Morton ([13], 
pp. 91-92). While they were employed with respect to Q, they are very readily 
applicable to the case of . With some very simple manipulation of his formulas, 
it is easy to derive the following theorem. 

Theorem 3.7. If q is an odd prime and q does not divide (4s — 5)(16s^-|-8s-|-5), 
there can he a cycle of length 4 for g{x, s) over if and only if there exists some 
solution z G Z of 



+ {3- 4s)z -h 4 = 0 (mod q) 
and a corresponding solution w G Z of 

— zw — 1 = 0 (mod q) 



such that 

{z{zw + 2)(z + 2)/q) = I . 

Furthermore, if such a cycle exists and q does not divide z® -I- 2z® -|- 4z^ -I- 6z^ — 
5z^ — 8z — 16, then s cannot he a generator for q. 

As an example, we give q = 23 and s = 40. We find z = 38 and w = 2 and 
the corresponding cycle is {36, 15, 39, 21}. 

4 Algorithms and Computational Results 

We have already mentioned that one would not expect to find many generators 
s for a given prime q. In fact, by using a computer, we found all the generators 
for each prime < 10"^. Let n{q) denote the number of generators for q and 

N{x) = #{q : n{q) = x,q< lO'^} . 

In Table 2 we present some values of N{x); note that N{x) = 0 if n > 8. 



Table 2. 



X 


N{x) 


X 


N{x) 


X 


N{x) 


0 


378 


3 


100 


6 


0 


1 


464 


4 


25 


7 


0 


2 


258 


5 


3 


8 


1 
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Thus, the average number of generators for each prime q < 10^ is 

iN{i ) / N{i) « 1.14. Incidentally, the value of q for which there are 8 gen- 

erators is q = 9767 with generators 1051, 1937, 2217, 2301, 3478, 3697, 5471, 6803. 

We used these generators to sieve out all the values of s < 1000 which must 
be a generator for some q < 10^. The remaining values of s are presented in the 
tableau below. 



2 3 


6 12 


20 ; 


21 30 42 


56 72 90 








105 


108 


no 


111 


128 


132 


156 


182 


195 


198 




206 


210 


213 


215 


240 


251 


272 


273 


287 


290 


293 


303 


306 


311 


338 


342 


356 


380 


381 








420 


437 


462 


471 


483 


495 












506 


525 


545 


548 


552 


570 


591 


593 








600 


612 


623 


630 


642 


650 


651 


656 


657 


675 




702 


713 


723 


726 


735 


740 


752 


755 


756 


768 


770 


800 


812 


821 


840 


857 


861 


870 










908 


912 


930 


936 


957 


965 


987 


992 


993 


996 





Of these remaining numbers, we see that 2, 3, 6, 12, 20, 21, 30, 42, 56, 72, 90, 
no, 111, 132, 156, 182, 210, 240, 272, 273, 306, 342, 380, 381, 420, 462, 506, 552, 
600, 650, 651, 702, 756, 812, 870, 930, 992 and 993 are all of the forms given in 
Theorems 3.2 and 3.4; hence, these 38 numbers can never be generators for any 
q. This leaves 54 numbers (< 1000) which may be generators for some prime 
g > 10^. If, in fact, the only values of s which can never be generators for any q 
are those given by Theorems 3.2 and 3.4, we would expect to be able to eliminate 
all of these 54 remaining numbers by increasing our limit on q beyond 10^. This 
means that we need to develop reasonably efficient algorithms for detecting when 
a particular s is a generator for a given q. 

Such an algorithm is implicit in Corollary 2.3.2, but as we have seen earlier, 
it is most unlikely that a given s will be a generator for a given q; thus, it is best 
to develop an algorithm that will quickly determine that s is not a generator for 
q (when this is the case). 

We note that since the average cycle length is expected to be the 

chance that it contains a zero node is y^S/irq which is very small. Indeed, if we 
examine all primes q such that 109 < q < 10® for s = 108 with the property that 
(s/q) = 1 and (4s -I- 1/q) = —1 and (4s — 3/q) = —1 and let m{x) = : x is 

the smallest number > 0 such that the cycle in the component beginning with 
X does not contain a zero node}, when m{x) yf 0 we get Table 3. 

Thus, to determine that s is not a generator for q we have the following 
algorithm: For x = 0,1,2,... up to a certain bound B we check whether the 
cycle in the component beginning with x contains a zero node. For this, we 
compute the sequences (gi(0, s))i>o and (g 2 i( 0 , s))i>o, and for each i = 1 , 2 ,... 
we check whether gi(0, s) = 52i(0, s) (mod q). This is expected to happen for 
i « 1.0308^ (Floyd’s method, see [15]). When this is the case, we know that 
3 i( 0 , s) is in the cycle for that i, so that we compute gi+i(0, s), gi+ 2 ( 0 , s), . . . until 
we find a minimal j such that gi+j(0,s) = 0 (mod q) or gi+j(0,s) = gi(0,s) 
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Table 3. 



X 


77l(x) 


X 


77l(x) 


0 


1185 


5 


1 


1 


11 


7 


1 


2 


4 


9 


1 


4 


2 


45 


1 



(mod q). If the latter happens first, we know that the zero node is not in the 
cycle, and s is not a generator for q. Otherwise, we compute for x = 1,2,... the 
sequences (gi(x, s))i>o and (g 2 i(x, s))i>o and, while doing this, check whether 
g 2 i-i = 0 (mod q) or g 2 i{x,s) = 0 (mod q). As soon as this happens, we 
know that x belongs to the same component as the zero node, which is in the 
cycle. If gi{x, s) = g 2 i{x, s) (mod q) for some i and gj(x, s) ^ 0 (mod q) for 
all J < 2 i, we know that x belongs to a component different from the one with 
the zero node. As soon as this happens for some x, we know that s is not a 
generator for q. Otherwise, s may be a generator. 

For example, for the 54 remaining values for s and the primes q between 10^ 
and 10® for which (s/q) = 1 and (4s + 1/q) = —1 and (4s — 3/q) = —1, with 
B = 100 we find that altogether 40 pairs (s, q) pass this test. Among these 40 
pairs, there are 15 cases where s indeed is a generator for q. When working with 
B = 1000, only 17 pairs (s, q) pass this test. Notice that, for given s, the test has 
to be applied only to about 12.5% of the 8363 primes between 10^ and 10®, since 
about 87.5% of the q are eliminated by checking the three Legendre symbols. 

The running time for each x = 0,1,...,B is proportional to the /9-length 
and hence is expected to grow with ^/q. Given s, for larger values of q we 
therefore use the following, faster algorithm: First we choose parameters jo 
and B. Now, given a pair (s,q) such that (s/q) = 1 and (4s -I- 1/q) = —1 and 
(4s — 3/q) = —1, we check whether TZ^ is empty. If this is the case, s cannot 
be a generator for q. Otherwise, we apply the criteria provided by Theorems 
3.5 and 3.7. If after that it is still possible that s is a generator for q, we com- 
pute TZa, . . . Only if TZj yf 0 for all k < jo and if ^ 

check for x = 0,l,2,...,i3 whether the cycle in the component beginning with 
X contains a zero node. For q € [10®, 10^°], a suitable choice for jo and B is 
jo = 30 and B = 2000. To illustrate the performance of this algorithm, for 
s = 108, 840 and n = 4, 5, 6, 7, 8, 9 we consider the least 10000 primes > 10” and 
let fc(n, s) = j/{q : {s/q) yf —1 or (4s + l/q) yf —1 or (4s — 3/q) yf —1}. By 
r 3 (n, s) we denote the number of those primes q not included in k{n, s) and for 
which TZo = 0. By 03 ( 71 , s) we denote the number of those q among the primes 
not counted so far for which we can prove the existence of a cycle of length 3 
using Theorem 3.5. Then, by 04 ( 71 , s) we denote the number of primes among 
the remaining values of q for which Theorem 3.7 establishes the existence of a 
cycle of length 4. By 030 ( 77 , s) we denote the number of those q that passed all 
tests so far and for which one of the TZ4, . . . , TZoo is empty and X)j=o < 9- 
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Then by b 2 ooo(n, s) we denote the number of those remaining q which do not 
pass the very last test that checks whether the component beginning with x 
(a; = 0, 1, , 2000) contains a zero node. Our sample results are shown in Table 
4. Here, the last column indicates how many primes q have passed all tests. These 
values of q are the only remaining candidates for which s can be a generator. 



Table 4. 



3 


k{n, s) 


r 3 (n,s) 


C3(n,s) 


C4(n, s) 


O 

CO 


62000 (u, s) 


survivors 








s 


= 108 








4 


5019 + 2506 + 1224 


516 


208 


103 


342 


82 


0 


5 


4993 + 2456 + 1299 


507 


238 


103 


327 


77 


0 


6 


4983 + 2512 + 1230 


524 


192 


119 


345 


95 


0 


7 


5019 + 2488 + 1265 


496 


192 


99 


350 


91 


0 


8 


5015 + 2518 + 1216 


489 


192 


122 


366 


82 


0 


9 


5039 + 2497 + 1208 


510 


204 


115 


360 


67 


0 








s 


= 840 








4 


4985 + 2534 + 1257 


512 


207 


108 


320 


77 


0 


5 


5042 + 2532 + 1211 


452 


214 


112 


360 


77 


1 (g = 182101) 


6 


4971 + 2517 + 1234 


487 


215 


115 


376 


85 


1 (g = 1053583) 


7 


5040 + 2525 + 1245 


442 


186 


145 


346 


71 


0 


8 


4990 + 2508 + 1245 


488 


214 


112 


354 


89 


0 


9 


4942 + 2558 + 1288 


478 


217 


98 


346 


73 


0 



To process the values of q which remain in the end, we choose some larger 
bound B' and check for x = 2001, . . B' whether all components that begin 
with X end in a cycle that contains the zero node. For q = 182101 in Table 4 
we find, for example, that the least positive number x that leads to a cycle that 
does not contain the zero node is a; = 3972, while for q = 1053583 this is the 
case for x = 80173. Hence, s = 840 is not a generator for these values of q. 

With the algorithm prescribed above we examined all primes q such that 
100003 < q < 2 ■ 10® for all 54 remaining values of s. For 36 values of s we found 
a prime q such that s is a generator for q. In Table 5 we give the corresponding 
values of s and q. Notice that for each s, we only consider the least possible prime 
q for which s is a generator. In the third and sixth columns we also indicate the 
minimal value for k such that X)?=o which also is the least k such that 

TZk+i is empty. 

In summary of Table 5, Table 6 shows for various ranges of q the number of 
values of s such that q is the least prime for which s is a generator. 

The remaining 18 values of s which are not a generator for any prime q <2 ■ 
10® are 108, 128, 290, 338, 495, 525, 545, 623, 630, 656, 675, 723, 735, 755, 770, 800, 
936, 987. Extrapolating from the data in Table 6, we expect that in order to find 
appropriate values of q for all of these values of s, we would have to examine 
all values of q up to at least 10^^. This seems to be a hopeless task at present 
because we know of no algorithm that executes in fewer than constant times q 
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Table 5. 



s 


9 


k 


s 


q 


k 


105 


97729 


863 


593 


92802097 


21661 


195 


1956979 


2658 


612 


134639 


833 


198 


820361 


1682 


642 


1643779 


3607 


206 


1746581 


4191 


657 


23035711 


11086 


213 


5994631 


6548 


713 


275657 


1242 


215 


25847 


508 


726 


14087 


286 


251 


89231 


582 


740 


16691 


374 


293 


56891 


835 


752 


23059 


444 


287 


8207041 


6542 


768 


12441217 


6044 


303 


29947 


363 


821 


40682441 


9916 


311 


631723 


2063 


840 


10830383 


6669 


356 


493853 


1470 


857 


98947 


888 


437 


34283 


359 


861 


29947427 


15438 


471 


20347 


369 


908 


2060843 


2695 


483 


31228199 


17327 


912 


141184027 


26912 


548 


58991 


708 


957 


1686701 


3379 


570 


493811 


1757 


965 


39191 


629 


591 


11369 


204 


996 


35053 


367 



Table 6. 



range for q 


# generators 


[hFAUFJ 


15 


[10®, 10®] 


6 


[io®,ioi 


7 


[ 10 ’’, 10®] 


7 


[10®, 2- 10®] 


1 



steps for proving that s is a generator for a certain prime q. Notice that, taking 
into account the 38 numbers of the forms given in Theorems 3.2 and 3.4 which 
cannot be a generator for any q, we therefore are left with 56 values of s < 1000 
for which a corresponding Shanks chain might have arbitrary length. 

The largest value of q that appears in Table 5 is q = 141184027, which is the 
least prime for which s = 912 is a generator. Here, = 9 for fc = 26912, 

with 77.26913 being the first set that is empty. The maximum value for #77^ is 
13198, while the average value for #77^ is 5245. With an implementation using 
the computer algebra system LiDIA [11], the computation of the 77^ took 4 
hours, 6 minutes and 54 seconds on a SPARC Ultra-60. For this computation we 
used the Tonelli-Shanks algorithm to compute the square root of a modulo q. 

We also devoloped another method to compute the square root of a modulo 
q that makes use of the continued fraction expansion of a/q. It works as follows. 
If g = 1 (mod 4), i.e., {—1/q) = 1, let A such that = —1 (mod q) and let 
n be a quadratic non-residue of q (which we can easily find by trial). If 9 = — 1 
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(mod 4), let A = 1 and n = —1. Now for 1 < a; < ^ we precompute tables of 
the Legendre symbols {x/q) and of the values G{x) and H{x), where 



G{x) 



y/x if {x/q) = 1 
if {x/q) = —1 



(mod q) 



and 



H{x) 



l/^/x if {x/q) = 1 
\/^/nx if {x/cj) = — 1 



(mod q) ; 



Here, \ft denotes either of the solutions, when they exist, of = t (mod y). To 
compute the square root of a modulo q when {a/q) = 1, we put xq = q, ri = a, 
Bi = 0, and B 2 = 1. For z > 2 we let 



Xi = rj _2 (mod r^.i) , 
qi-i = rj _2 (div r^.i) , 
Bi-\-i — qi—\Bi ~\~ B^—i , 



until we find a minimal i such that Bi^i > ^Jq. Then 

(-l)Vi_i = aBi (mod q) 

and 

{a/q) = {Bi/q){ri_i/q){-l/qy , 

where Bi, ri_i < ^ (see [20]). It is now easy to verify that if {a/q) = 1, then 

^ ( {G{n_i)H{By)^ {modq) if 2 | z 
^ ( {XG{ri-i)H{Bi))‘^ (mod q) otherwise . 

Thus, once Bi and have been found, {a/q) can be easily determined and, if 
{a/q) = 1, then ^/a (mod q), can be computed simply by table look-ups and 
multiplication modulo q. It turns out that for large values of q, this method 
speeds up our algorithm by about a factor of 2. For example, to prove that 
s = 912 is a generator for q = 141184027 with the new method took only 2 
hours, 6 minutes and 3 seconds, on the same machine as before. 

We made a special effort to find values of q for which s = 108 and s = 290 are 
generators. This was without success - we only found that s = 108 and s = 290 
are not generators for any prime q < 10^*^. Moreover, because of Theorem 3.4 
we also tried to find other values of q for which s = 1 is a generator. We found 
that 1 is a generator for 3, 7, 23, 19207 and no other prime < 2.1 • 10®. 

While for most of the primes q we can determine very rapidly that a given s 
is not a generator for q, once in a while we run into a value for q which requires 
much more effort. For example, for s = 1 and q = 1523053897, the least positive 
X such that the component beginning with x ends in a cycle that does not contain 
the zero node is a; = 2765848; this component consists of 962 elements, and the 
corresponding cycle length is 26. On a SPARC Ultra-60, it took 14 days, 16 hours 
and 35 minutes to find this value of x. To determine that s is not a generator 
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for q by considering the sets TZi, we have to compute TZi up to z = 121092 (i.e., 
"7^121093 is the first set that is empty) until we find that the component that 
contains the zero node consists of only 1523052733 = q — 1164 elements. This 
computation took 1 day, 9 hours and 56 minutes on a SPARC Ultra-60. 

In Table 7, we list for s = 1, s = 108 and s = 290 those primes q > 10® 
{q < 2 ■ 10® for s = 1, and q < 10^® for s = 108, 290) that have passed all 
tests described prior to Table 4, and for which all a; = 0, 1, . . ., 10000 end in 
a cycle that contains the zero node. The respective cycle lengths are indicated 
in the second column of Table 7. However, for all these primes we eventually 
found a second component: in the third column we give the least x that ends 
in a cycle that does not contain the zero node, while the last column shows the 
corresponding cycle length. 



Table 7. 



1 1 


1 


1 X 


1 1 


1 S = 1 1 


1055114873 


2 


53870 


70 


1121788583 


2 


12934 


278 


1307586407 


2 


11064 


41 


1523053897 


2 


2765848 


26 


1 s = 108 1 


1361042663 


35227 


99379 


7 


3323409469 


130529 


130529 


905 


3570912959 


28182 


1574734 


24 


3945934931 


79534 


694699 


13 


5626917623 


159574 


47016 


373 


1 s = 290 1 


1492251769 


78613 


69682 


123 


2258948569 


97638 


18895 


1198 


2262261047 


26405 


62921 


18 


3870012343 


153977 


190103 


24657 


4696002397 


19510 


29918 


350 


5824284551 


42637 


37610 


499 


7865621479 


98782 


42567 


379 


8273290073 


174261 


1627204 


44 



Another exceptional pair (s, q) is given by s = 545 and q = 16251619: Here, 
we find that the component that ends in a cycle with the zero node consists of 
q — 12 elements, while there is a second component that consists of 12 elements 
and ends in a cycle of length 5. The least positive number x that belongs to that 
second component is a; = 4048245. 
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Abstract. This article presents algorithms for computing discrete loga- 
rithms in class groups of quadratic number helds. In the case of imaginary 
quadratic helds, the algorithm is based on methods applied by Hafner 
and McCurley [HM89] to determine the structure of the class group of 
imaginary quadratic helds. In the case of real quadratic helds, the algo- 
rithm of Buchmami [Buc89] for computation of class group and regulator 
forms the basis. We employ the rigorous elliptic curve factorization al- 
gorithm of Pomerance [Pom87], and an algorithm for solving systems 
of linear Diophantine equations proposed and analysed by Mulders and 
Storjohami [MS99], Under the assumption of the Generalized Riemami 
Hypothesis, we obtain for helds with discriminant d a rigorously proven 
time bound of L|d| [i, |\/2]. 



1 Introduction 

Currently the best available algorithms for extracting discrete logarithms (DL) in 
class groups of number helds proceed by computing the class group — generators 
and relations — hrst, and continue from there by linear algebra to calculate the 
DL (cf. e.g. [BD90]). This seems more work than necessary, since the class group 
problem appears to be more dilhcult than the DL problem. Indeed, we will show 
in this article that it is possible to evade the necessity of computing the class 
group hrst. On the contrary, the methods applied can be extended to compute 
the class group. 

Since Gauss, there has been continuous interest in computing class groups. 
Comparatively recently, the interest has also turned to the question of how to 
compute them efficiently. With [HM89], and [Buc89] we would like to mention 
two papers which made a breakthrough by proving that this calculation can be 
done in time subexponential in the size of the discriminant of the held: [HM89] 
did this for imaginary quadratic helds, [Buc89] for general number helds. Here, 
we will employ the methods of these papers, and sharpen their results. 

The point of view of this article, however, is cryptographic, and we will 
focus on giving rigorous time bounds for the DL problem. The cryptographic 
interest in this problem arises from several proposals for using class groups as the 
underlying algebraic structure for cryptosystems, see e.g. [BW88], and [McC89]). 
* research supported by the DFG 
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III finite fields — the algebraic structure for which DL-based cryptosystems were 
first proposed — discrete logarithms can be computed in time Lg[^,l], where 
q is the size of the field, and, for real numbers x > e, a, b, we set as usual 
Lx[a, b] = exp (6(loga;)“(logloga;)(^““^). 

When we try to reproduce this result and the methods which were used to 
obtain it in the case of class groups we have to struggle with the fact that the 
size of the group we are working with is not known beforehand. It is therefor 
tempting to overcome this dilficulty by first computing this size or even the 
structure of the group at hand, and then use standard procedures for general 
groups to solve the DL problem. 

Our approach evades this necessity. We do apply the usual index calculus 
approach, as proposed in the case of imaginary quadratic fields by Hafner and 
McCurley [HM89] together with an improvement already proposed by Hafner 
and McCurley themselves, namely the use of the rigorous elliptic curve factor- 
ization algorithms introduced by Pomerance [Pom87]. However, the final linear 
algebra step, which originally involved the calculation of the Hermite normal 
form of the relation matrix is replaced by a direct computation of the discrete 
logarithm on the basis of the relation matrix, employing the recently proposed 
algorithm for solving Diophantine linear systems by Mulders and Storjohann 
[MS99]. 

As a result of these improvements we prove rigorously that our algorithms 
(DL extraction in the imaginary and real quadratic case, class group computation 
in the imaginary quadratic case) can be executed in time L|d| [^, |-\/2+o(l)], with 
the sole premise of the Generalized Riemann Hypothesis (GRH) . In comparison 
to McCurley [McC89] who obtained the same time bound for the computation 
of the class group of an imaginary quadratic field we do not have to make any 
assumptions on the behavior of intermediate results. 

For background information on quadratic fields, and their class groups we 
refer the reader to [LP92] and [Coh93]. 

2 The Imaginary Quadratic Case — Description of the 
Algorithm 

Let K be an imaginary quadratic field of discriminant —d. For ease of calculation 
we will work with binary forms instead of ideals. We denote hy Cl {—d) the set of 
PSL 2 (Z)-equivalence classes of positive definite primitive binary quadratic forms 
of discriminant —d, with group structure induced by Gaussian composition. The 
class with representative / will be denoted by [/]. We will omit the brackets, 
however, where no confusion will arise. 

The IQNF-DL problem. Given two forms g and h the task is to decide whether 
there exists an exponent I G Z such that [gY = [h], and, if the answer is positive, 
to find one. (Since this task is trivial if g is in the principal class — which can 
easily be checked by reduction — we will assume in the foil wing that it is not.) 

For the set-up of the index calculus we first need a large set T of prime 
forms as a factor base. Let V-d = {p prime | (^) = 1}. It is easy to determine 
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Algorithm 1: DL-algorithm in CL(—d) 

Input: Discriminant —d of an imaginary quadratic field K, 
two form classes [<?], [h] £ CL{—d), error probability e 

Output: either natural I such that [<?]* = [h] or UnDef, 

meaning that with probabihty 1 — e there is no such 1. 



1qDl(— d, g, h) 

1. Construct the factor base T'. 

^ ■■= {[/] I / = (P> b, -),p € V-d,P < Ld{\, ^)} 

2. Construct the generating set Q: 

5 :={[/] I / = {P,b, -),P £ V-d,P < 61og^ d} 

3. Construct the extended factor base S: 

S := U Q U {g,h} 

4. foreach f € £ 

riC) ~ 1 qRelation(/, 2nd, Q U {/, g}, n^d) 

5. for i:=l to 3nlogd — 31oge 

:= 1qRelation( 1, 0, 5, d^) 

6. Collect relations and into matrix A =: (^) with first row a 
containing exponents of g 

7. DiophantineSolver(A', ei , e/2) =: (y, d) 

8. if A'y = ei then return I := a ■ y 

9. else return UnDef 



whether any given p belongs to V-d- Let no = \Ld{^, . We collect into T 

the first hq prime forms (p, b, •) with p £ V-d- 

In order to produce random forms we need a set of generators of Cl{—d). 
Due to a theorem by Bach [Bac90] it sullices to use the set ^ = {[/] | / = 
(p, 5, -),p prime,p < clog^d} where c can be chosen to be 6 in the case of 
quadratic fields, and 12 for general number fields. 

Finally, we represent Cl{—d) as a quotient of the lattice spanned by the union 
£ = T yj Q yj {g,h\ , which we will call the extended factor base. Remember that 
for large d Q C V so that we really add only g and h to T. Let n := card£ = 
max([Ld(5, ^)l,61og^ d) + 2. 

We have the obvious group homomorphism 

fee 

Its kernel A is a sub-lattice of full rank since Cl{—d) is finite. We construct a set 
Ti of m := relations u £ which generate A for sulliciently large d with 

probability 1 — e/2, where the error probability for the total algorithm e is given 
in advance. To this end we follow the procedure of [HM89]: we compute random 
form classes, and try to factor their reduced representatives over our factor base 
V. 
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Algorithm 2: Generation of relations 

Input: form /, exponent u, set of generators 7t, radius r 

Output: relation v = (ve)e € £ s.th. \vf — a| < logd, and for e ^ f \ve\ < 
r + logd if e e or else \ve\ < logd 



IqRelation(/, a, H , r) 

1 . repeat 

2. Draw random {ue)e<^H from with the uniform distribution 

3. Let f' = (a, b, c) be the reduced form in the class /“ Heew ■ 

4. until attempt to factor a with Algorithm 7.2 out of [LP92] is successful 
where we choose y := Ld{\, -^) as upper bound for divisors of a . 

5. Find with method (2.8) of [LP92] {t e)e^:F 

(a,b,c) = 

and let te = 0 for e £ £\T. 

6. return (se)ee£, where 

{ U + Ue—te if e = /, 

Ue — te if e G e 7 ^ /, 

— te if e £ £\H. 



In order to generate the first n relations we start as in [Sey87] with large 
factors where / runs through S, multiply each with random forms from 

(the image under (j) of) a box in of radius ri^d until we hud a multiple 

that can be factored by the elliptic curve method. Lenstra and Pomerance [LP92], 
Theorem 8.1., show that these multiples can be factored with probability larger 
than Ld{^, note that its preconditions are fulhlled since we assume the 

GRH (cf. the Remark following the proof of Theorem 8.1.). The n relations found 
generate already a full-ranked sub-lattice Aq of A because the relation vectors 
can be arranged to form a diagonally dominant matrix. In contrast to [HM89], 
we do not need to compute the determinant of Aq. 

The rest of our relations are then chosen as in [HM89] to be approximately 
evenly distributed in Aj A^ by drawing random forms from (the image of) a box 
in if of radius (P . We do not check whether 7f indeed generates A since by 
adjusting m we can achieve that it will do so with predetermined probability 
e/2, (cf. [HM89]. If d has more than 10 decimal digits Snlogd— Slog (e/2) will 
sulhce.) 

Assume for the moment that H indeed generates A. Let A be the matrix 
whose column vectors are the v GH. We may arrange the rows of these vectors 
in such a way that the entries corresponding to the exponents of g and h appear 
in the hrst and second row, respectively. Then the DL problem is solvable if and 
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only if A contains some vector of the form (1, 1,0,... 0)^ . Due to our assumption 
that 7i generates A this happens in turn iif 

A'y=(l,0,...0f (1) 

is solvable, where A' is obtained from A by striking out the first row a. If y is a 
solution of the Diophantine linear system (1), then we have = [h] for I = a-y 
since (i,l,0,...,0)^ G A. Note that the value I thus found is not necessarily 
minimal. 

The algorithm DiophantineSolver we apply for the solution of (1) was 
recently developed by Mulders and Storjohann [MS99]. Given as input a system 
Ay = b that is solvable over Q, and some error probability 6 Diophantine- 
Solver yields with probability 1 — <5 a pair (y, d) with natural d, and integral y 
whose entries are bounded in bit length by O'^(n) such that 

Ay = db, (2) 

and the output d is minimal among all pairs fulfilling (2). The remainig cases 
which occur with probability 5/2 each are that d is not minimal, or no solution is 
given at all. The algorithm is probabilistic, and even when successful may return 
diiferent y in diiferent runs. At any rate, (1) is solvable if a pair (y, d) with d = 1 
is returned. 

There remains the unlikely case that we have not found a complete lattice of 
relations. In this case (1) will not necessarily be solvable, even though the DL 
problem might be. Thus if the check in the last step of Algorithm 1 is negative 
(the case labeled UNDEF in the listing of the algorithm) then we know that 
either (a) [h] does not lie in ([g]), or — with controlled, small probability — (b) 
H does not generate A/Aq, or (c) DiophantineSolver returned no solution 
or one with inaccurate denominator. If one is willing to invest some more time 
(within the same asymptotic time constraints) then it is possible to certify that 
indeed case (a) precludes finding a relation (i, 1,0,..., 0). We will deal with this 
task in section 4. 

3 Running Time Analysis 

Steps 1 to 3 take at most 0(n) bit operations (cf. [McC89]). 

For an estimate of the running time of steps 4 and 5 we apply the analy- 
sis in [LP92]. The collection of relations takes expected time ■ 

Ld(i,^) = -I- o(l)) multiplied by the time needed for the actual 

factorization which can be absorbed into the o(l) term. This term is eifectively 
computable on the basis of the data given in [LP92], and our estimate of m. 

The solution of the system in step 9 uses an expected number of = 

+ o(l)) bit operations. For the analysis of the perturbation method 
see [MS99]. The final step of our algorithm takes time 0{n). 

In consequence we have the following 
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Theorem 1. (GRH) There exists a probabilistie algorithm that deeides for dis- 
eriminant —d, and forms g,h ^ Cl{—d) with error probability e given in advanee 
and independent of d, g, h whether there exists an I sueh that g^ ~ h, and eom- 
putes some I in expeeted time Ld{^, + o(l)). 

4 Non-solubility of the DL Problem and the Computation 
of the Class Group 

There remains the blemish on Algorithm 1 that it is not able to certify that there 
is no solution of the DL problem. In order to be able to do this we have to verify 
that the set of relations found in Algorithm 1 generates the full relation matrix. 
As a side product we compute the class number, and may also optionally extract 
the structure of the class group of the given imaginary quadratic held. Once we 
have established that we have the full relation lattice we will use the algorithm 
proposed in [GLS98] to certify inconsistency of the diophantine system (1) over 
Z. 



Algorithm 3: Verihcation of lattice of relations 

Input: discriminant —d of imaginary quaratic held K; 

relation matrix A of some (extended) factor base S containing gen- 
erating set 0; the first cardC/ rows of B are occupied by entries cor- 
responding to exponents of elements of Q 
Output: True if the columns of B generate a full relation matrix of S, 
Error else 



1. Using the class number formula compute a rational number h with 
h/2 < h(—d) < h 

2. Compute C := Triang(A, cardC/, e). (With probability 1 — e an upper 
triangular C will be returned which will generate the full relation lattice 

of e?). 

3. if Triang returned Error or det C does not he in the range predicted 
by step 1 then return Error 

4. return True. 

5. [Optional] Compute the Smith normal form of C to obtain the structure 
of Cl{-d). 



Let A' be the lattice generated by 7f . In order to check whether A' coincides 
with the full relation lattice A we calculate the determinant of A' as the deter- 
minant of the (essential part) of the HNF of a matrix B whose columns are the 
vectors in 7f . This time we arrange the rows in such a way that the exponents 
of elements in Q come hrst, and enumerate Q =: {gi, . . . , pq} according to the 
row numbers in B. Obviously, the essential part of the HNF of B is restricted 
to these rows. 
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Algorithm 4: Triangularization of matrix with small essential part of HNF 

Input: n X m matrix B of full rank; r highest number of a row in the HNF 
of B with diagonal entry not 1; error probabihty e 

Output: triangular Co with C := = BU for some U G Gl{m,h) 



Triang(H, r, e) 

1. For fc n to n — r + 1, let be the k x m matrix obtained from 

(k) 

B by striking out the first n — k rows. Let e\ be the fc-dimensional 
column vector ( 1 , 0 ,... , 0 )^. 

2. Let d^'"'):=DlOPHANTlNESoLVER((H^'"\ 61 *"^ e/r^)). Collect the 

m-dimensional column vectors into the matrix Y. 

3. if none of the calls to DiophantineSolver returned Error then 
return Co, the matrix obtained by striking out all but the first r rows 
of the product AY. 



Let C be the HNF of B with zero columns removed. Let further be 

the diagonal elements of C. Then 

Cfc = min{c | ( *, . , * , c, 0, . . . , 0)^ € A'}. (3) 

(fc — 1)— times 

(If A' = A then we have also Ck = min{c \g%& (gi, . . . , gk-i)}.) The Ck are easily 
found with the methods that were already employed in section 2: Strike out the 
corresponding number of rows of B which will yield some B^^'> , and try to solve 

where = (1,0,... ,0)^ of the necessary dimension. From Diophantine- 
Solver we will get with probability 1 — 5/2 some pair {y^^\ dk) where dk with 
conditional probability 1 — <5/2 is minimal, and, thus, equals Cfc. This calculation 
needs to be done only q = 6 log^ d times, since Q generates the class group, and, 
hence, the remaining Ck for k > q are 1. 

It remains for us to collect the into a transformation matrix Y, 

multiply B with Y, and read off the dk- Their product is the sought determinant 
h' of A' . By comparing h' with bounds for h := h{—d) = det A which we may 
obtain from the analytic class number formula as e.g. in [McC89] we can now 
decide whether A' = A. 

In order to limit the error probability of the verification algorithm to our 
e we need to adjust appropriately the error probability <5 we set for each call 
to DiophantineSolver. A rough, but sufficient estimate would be <5 := e/q^ 
where q = card G = 6 log^ d as above. This has little effect on the total run- 
ning time of the algorithm since the complexity of DiophantineSolver grows 
logarithmically with <5. The total time for the verification of A' = A is hence 
q ■ O (log log d) • 0(n^+°(i)) which is still 0(n3+°b)). 
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Considering that the calculation of the Smith normal form of a matrix of size 
q takes at most 0{q‘^) operations we have as a by-product the following 

Theorem 2. (GRH) There exists a Las- Vegas algorithm that eomputes the elass 
number, and the strueture of the elass group of an imaginary quadratie field with 
diseriminant —d with error probability e given in advanee and independent of d 
in expeeted time Ld{^, + o(l)). 

Once we are assured that (1) does have a solution whenever the DL-problem 
is solvable we turn to the certihcation algorithm by Giesbrecht, Lobo, and Saun- 
ders [GLS98]. Their algorithm CertifyZInconsistency needs square matrices 
with rational solutions as input. By replacing Q with U {g} in the above trian- 
gularization process we obtain a suitable small (g -f 1) x (q+ l)-matrix C. This 
matrix is triangular, and there is an (easily computable) triangular unimodular 
matrix U such that CU is in Hermite normal form. This is also the essential part 
of the HNF of A' which can be written as 



id := 



fO CU 0\ 

Vo 0 l) 



Thus, a solution of Cy = e\ would yield one for (1) which we assume not to exist. 
Furthermore, a certihcate for non-solubility of Cy = e\ can easily be translated 
into one for (1). Indeed, CertifyZInconsistency yields a prime p and a row 
vector u s.th. uC = 0 mod p, but u ■ ei = ui 0 mod p. Extending u with 
zeroes to dimension n — 1 yields the vecor u' which annihilates H, and therefor 
also A'. Since its hrst entry is of course still nonzero we have u'A = 0 mod p, 
but u' ■ ei ^ 0 mod p, i.e. u' is a certihcate for non-solubility of (1). 

Summarizing, we have found an algorithm for certifying the non-solubility of 
the DL-problem. 



Algorithm 5: Certify non-existence of DL in Cl{—d) 

Input: discriminant — d of an imaginary quadratic held if; 

n X m matrix A of relations between elements of an extended 
factor base, the hrst q + 2 rows of which contain the entries cor- 
responding to argument h and base g of the logarithm, followed 
by the members of a generating set Q. 

We assume there is no y s.th. Ay =(*,1,0,..., 0)^ 
error probability e 

Output: n X 1-vector u with nonzero second entry such that uA = 0 



1. Remove the hrst row form A to obtain B, and set 
C = TRlANG(ii, q + l,e) 

2. Let u = CertifyZInconsistency(C', ei, e), and let 

U' = (0,Ul, . . . ,Mq+l,0, ... ,0) 

3. if u'A = 0 then return u' 
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This algorithm may already be applied when Algorithm 1 returns UnDef. 
In this case we need to limit the time spent on CertifyZInconsistency, and 
pronounce failure whenever no certificate is returned in this time. With this 
approach we have an algorithm of Las Vegas type that decides the DL-problem, 
and whose output can rapidly be checked whether the DL-problem is solvable 
or not. 



5 The DL-Problem in the Real Quadratic Case 

The real quadratic case dilfers from the imaginary one by three elfects: (a) class 
groups of real quadratic fields are usually small, (b) real quadratic fields have 
non-trivial units, and (c) the notion of a reduced form, and the algorithm of 
reducing a given form dilfer somewhat from the imaginary quadratic case: Most 
significantly, there is not one reduced form in each class, but a cycle of forms 
which is traversed by successive reduction. 

Factor (a) leads us to use the the ideal group instead of the class group as 
the setting for our problem. 

The RQNF-DL problem. Let if be a real quadratic field of discriminant d. 
Given two ideals g and [} in Ik the task is to decide whether there exists an 
exponent 1 G Z, and some a G K such that 

= a - f}, 

and, if the answer is positive, to find one pair {I, a). 

For ease of calculation, we will continue to work with binary forms as well 
as ideals. Thus, we will switch back and forth between representing the class 
group by ideals or forms. Thus Cl{d), the ideal class group, is identified with the 
set of PSL 2 (Z)-equivalence classes of indefinite primitive binary quadratic forms 
of discriminant d modulo relations (a, b, c) — (—a, b, — c), which is endowed with 
the group structure induced by Gaussian composition. As before, the class with 
representative / will be denoted by [/] . Additionally, we adopt the notation used 
in Chapter 5 of [Coh93], and will denote the map from forms to ideals, or ideals 
to forms, and their corresponding classes by 4>fi , 4>iF,ipFi respectively. 

Factor (c) forces us to adapt our algorithm: we need to specify the number 
of reductions which are to be performed at each composition, and a method 
that ensures the (approximately uniform) selection of a reduced representative 
in each form class. The first is easily done by requiring the minimal number of 
reductions that is needed to arrive at a reduced form. The latter is achieved 
by the algorithm Reach proposed by Abel in [Abe94]. (Given as input some 
uniformly distributed y from a region in M that is sulficiently large in comparison 
with the regulator, and some reduced ideal, her algorithm yields in polynomial 
time a random reduced ideal in the same class, and a relative generator of the 
pair which is of size close to y.) 
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Algorithm 6: DL-algorithm in CL{d) 

Input: Discriminant c? of a real quadratic field K, 
two ideals g, 1} G Ik, error probability e 

Output: either natural I such that = [}, some B C K, and a vector 
of natural numbers each of bit length bounded by 

0(n), where n := max(Lci(i, ^),61og^(d)) + 2 such that 



4>Fi{gy 




• 4>Fi{h) 



or UnDef, meaning that with probability 1 — e there is no 
such 1. 



RQDL(d,fl, [}) 

1. Construct the factor base T\ 

•^ :={[/] I / = (P, b, •), [I) =1,P< La{\, ^)} 

2. Construct the generating set Q\ 

G ■= {[/] I / = ip,b,-), (l) = l,p< clog^d} 

3. Construct the extended factor base E\ 

S ■.= T\JQ\J {(/'/F(fl), 

4. foreach f € S 

■= RqRelation(/, 2nd, tfu {/,5},n2rf) 

5. for i:=l to Snlogd — 31oge =: mo 

:= RqRelation( 1,0, d^) 

6. Collect relations and into matrix A =: (^) with hrst 
row a containing exponents of h 

7 . DiophantineSolver(A', ei, e/2) =: (y, d) 

8. if A'y ^ d then return UnDef 

9. Let I = a • y 

10. Let B := [J^^^Bf U reindex the and 5^*^ with 

index set B, and set S/s := Y.fe£ + YXh 

11. return (1,R, (sy)/ 3 gB) 



Furthermore, we will modify our algorithm to yield the additional information 
necessary to calculate the relative generator a. Each time we arrive at a relation 
V we record its generator i.e. the generator of the ideal Y\e4’Fi{eY^ ■ The 
vector y = (y„) giving a combination of relations that equals a “DL relation” 
(*,1,0,... ,0) can now also be interpreted as an exponent vector for calculating 
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We will not actually calculate the and a explicitly since this would take 
exponential time and space to complete. These quantities will be expressed in 
compact form as pairs of two vectors containing bases and exponents, respec- 
tively. Note, however, that in practical cryptographic circumstances a will have 
a short representation which we are able to compute efficiently on the basis of 
this vector pair by an approximation technique. 

We will give the “real’ versions of Algorithm IqDl, and IQ Relation on 
pages 590, and 592 respectively. 

When reading algorithm RqDl it is best to imagine that the matrix com- 
posed in step 6 indeed contains one more row in which the generators of the 
relation are recorded: 



A := 



if) (1) i^o) 

Ve ■ ■ ■ Vy ■ ■ ■ Ve 



• • CXf • • • CXi • • • CXmo J 



If the call to DiophantineSolver in step 7 returns a solution (y, 1) then A-y = 
(1,1,0,... ,0,Qf)^, where operations in the last row are in the multiplicative 
group K* . 

We must convince ourselves that the a* which are returned in compact form 
by Rq Relation do indeed generate the principal ideal associated with the rela- 
tion generated. For this one needs to follow the composition of forms in step 8. 
Each time, the reduction operator is applied on a form / = (a, b, c) we gather as 
factor the relative generator j3 = (6-1- ^/{d))/2a for which 4>Fi(p{f)) = P4>Fi{f)- 
Doing this successively yields 

l[^Fi(.er‘=l[P*yFi([[e^‘) (5) 

The last equality holds since the product on the right is equivalent to 1 not only 
modulo PSL2(Z) but by construction already modulo Too, the kernel of 4>fi- 

The time needed for the execution of the real quadratic algorithms does 
not differ asymptotically from those of their imaginary quadratic counterparts 
despite the additional bookkeeping involved. (There is of course a slight change 
in the o(l) term.) Note in particular that RqRelation generates in steps 2 to 
4 random reduced forms with a sufficiently uniform distribution, and that the 
probability that random reduced forms can be factored does not differ between 
the positive definite, and the indefinite case. 

In consequence we have the following 

Theorem 3. (GRH) There exists a probabilistie algorithm that deeides for dis- 
eriminant d, and forms f) G Cl{—d) with error probability e given in advanee 
and independent o/d, g,[} whether there exists a, I sueh that = a • [), and 
eomputes some I, and some a in expeeted time Ld{\, + o(l)). 
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Algorithm 7: Generation of relations among generators of Cl{d) 

Input: form /, exponent u, set of generators H, radius r 

Output: relation v s.th. \vf — u| < logd, and for e ^ / \ve\ < r + logd 
if e G H, or else |fe| < logd 

set B C K, and vector {sj3)i3^B s.th. Yie&e ~ (ri/3eB 

RqRelation(/, u, H, r) 

1. repeat 

2. Draw random {ue)e^n from with the uniform distribution 

3. Let /' := /“rieGK®”°- course of the composition of 

two forms, the reduction operator p is applied the minimal 
number of times to reach a reduced form.) 

4. Select a random reduced form f" = {a",b",c") in the cycle 
of f by choosing some y G [— d, 0], and, employing Reach 
as dehned in [Abe94] to hud 7, and f" s.th. ln\^\ ~ y, and 
4>Fi{f") = (7) • 4>Fl{f')- 

5. until attempt to factor a" with Algorithm 7.2 out of [LP92] is 
successful where we choose y := Ld{\,^) as upper bound for 
the divisors of a" . 

6. For all e G IF, e = {p,bp,-) let te be s.th. || a", and b = 
sign.{te)bp mod 2p. Thus 

(a",6",c")= He*'- 

Let further te = 0 for e G F \ IF. 

7. Compute the exponents (fe)eef! where 

{ u + Ue-te if e = /, 

Ue-te if e G 7d, e /, 

— te if e G F \ 

8. Recompute Hee^ time keeping track of the factors intro- 

duced by the reduction steps along the route: for every pair of 
forms ei, 62 to be composed we obtain /3ei,e2 s.th. 

4>Fi{e\) • 4>Fi{e2) = /3(ei, 62 ) • 4>Fi{e\ • 62 ). 

Collect the (3 found together with 7 from step 4 into B counting 
multiplicities with the exponents sp. 

9. return (t>e,R, (s/3)/3gb) 
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Contrary to the imaginary quadratic case, we are not able yet to certify 
within the same time constraints that some [) does not lie in the subgroup (g) C 
CL{d). In order to verify whether the lattice spanned by the found is a 
complete relation lattice we need a bound on the class number. Here, the analytic 
class number formula delivers only bounds on the product of class number and 
regulator. Thus we would need to compute the regulator first. To the best of 
our knowledge, the algorithms currently available to do this on the basis of the 
data already accumulated are too slow, with time constrained in our set-up by 

5/2 + 0(1)). 

[One such algorithm would be a speed up of Buchmann’s algorithm [Buc89] 
combining ideas of Abel [Abe94] with a recently developed algorithm by Maurer 
[MauOO] that enables us to rapidly calculate a fundamental unit given a gener- 
ating set of units. In [Abe94], Abel shows following closely [Buc89], and [BK92] 
how to generate sufficiently many relations so that the logarithms of their rel- 
ative generators span a lattice in M that contains the fundamental unit, and 
how to control the logarithms of these relative generators in the course of the 
regulator computation. 

However, it seems very likely that a further, minor speed-up of the linear 
algebra step of the outlined algorithm should allow to sharpen the time estimate 
to match that of the imaginary quadratic case.] 

6 Conclusion 

We have analysed the DL-problem in two environments for which there have been 
proposed cryptographic (e.g. key exchange) protocols based on the difficulty of 
this problem. The asymptotic behaviour of the algorithms presented here though 
considerably better than that of previous algorithms lags still behind the best 
rigorously analysed one for the similar problem in the multiplicative group of 
finite fields, or those for factoring integers. This may indicate that algorithms 
which work in groups whose size is difficult to compute are harder to break than 
those in groups of known size. 

It remains a very interesting problem whether this gap can be closed, i.e. 
whether there are algorithms which are amenable to rigorous analysis which 
compute the DL in class groups in time Ld{\, 1 + o(l)). Moreover, it would be 
very interesting to see whether methods similar to the general number field sieve 
can be applied in this environment in order to find an algorithm that at least 
heuristically matches the state of the art in factoring integers. However, if that 
turns out to be beyond the reach of current methods, and ideas, we would have a 
strong indication that algorithms in class groups would indeed deliver enhanced 
security over USA- type algorithms. 

There also remain more straightforward extensions of the current work. We 
would just like to mention on the easy side the calculation of the regulator of 
real quadratic fields, and the extension to non-maximal quadratic orders, on the 
more difficult one the removal of the assumption that the Generalised Riemann 
Hypothesis holds, and the extension to class groups of fields of higher degree. 
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Abstract. We present an asymptotically fast algorithm for the compu- 
tation of the greatest common divisor (GCD) of two Gaussian integers. 
Our algorithm is based on a controlled Euchdean descent in that the 
operands are not reduced too much in each Euchdean step. To compute 
a descent of n bits, the algorithm recursively calculates two descents 
of approximately n/2 bits each with short operands and transfers the 
thereby calculated cofactors to the original operands. Overall, this algo- 
rithm achieves a time bound of 0(n(log n)^ loglog n) bit operations for 
operands bounded by 2" in absolute value. 

Eurthermore, we show that the biquadratic residue symbol of two boun- 
ded Gaussian integers can be computed as fast as the GCD can be cal- 
culated (except for a larger constant hidden in the O-notation). This 
result is achieved by first calculating the GCD of the operands, and then 
using the sequence of quotients from the Euchdean descent to compute 
the biquadratic residue symbol. 



1 Introduction 

This paper deals with an asymptotically fast algorithm for computing the great- 
est common divisor (GCD) of two Gaussian integers, i. e. complex numbers with 
real and imaginary part both integer. The main idea is that one does not op- 
erate with the whole operands all the time, but instead uses prefixes of the 
operands to calculate a Euclidean descent, i.e., a sequence of Euclidean steps 
where the remainders are greater than a size bound. This technique is adapted 
from GCD calculation in Z; it is called controlled Euclidean descent, first used 
by Schonhage [Sch87] for fast GCD calculation in Z. We will discuss our new 
GCD algorithm for Z[f] in detail and will prove that it achieves a time bound of 
0{n{logn)^ log log n) bit operations for Gaussian integers bounded by 2” in ab- 
solute value. After this we give a brief overview how to calculate the biquadratic 
residue symbol in a fast manner. This is an application of the fast GCD calcu- 
lation in Z[f]. Einally, we look at some practical running time of several GCD 
algorithms in Z[f]. 

Eirst we review previous work on GCD algorithms in Z and Z[f]. We see that 
some of the techniques used to accelerate the GCD calculation in Z can be used 
to accelerate the GCD calculation in Z[f]. Thus it is helpful to also discuss the 
development of GCD algorithms in Z. 



W, Bosnia (Ed,): ANTS-IV, LNCS 1838, pp, 595—613, 2000, 
Springer- Verlag Berlin Heidelberg 2000 
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Since Euclid (cf. [Hea56, Book VII, Propositions 1-3]) it is well-known how 
to calculate a GCD in TL in polynomial time. However, the computation with 
the so-called Euclidean algorithm is not optimal. Lehmer [Leh38] developed a 
technique for faster GCD calculation that uses as long as possible only single- 
precision prehxes of the operands in the Euclidean steps. With every Euclidean 
step, the cofactors are also updated such that the calculated reduction can be 
transferred to the whole operands. This algorithm speeds up the standard Eu- 
clidean algorithm because the rounded quotient of two random integers will be 
less than 1000 approximately 99.856 percent of the time, cf. Knuth [Knu98, 
4.5.2, high-precision calculation]. Therefore it sulhces to calculate most of the 
Euclidean steps in single-precision. Sorenson [Sor95] rehned this algorithm and 
achieved a running time of 0(n^/logn). Due to Stein [Ste67] is the so-called 
binary algorithm. This algorithm uses another idea for speeding up the GCD 
calculation in Z; it is based on the fact that the difference of two odd inte- 
gers is even and thus divisible by two. At hrst, the operands are made odd 
by dividing them by two as long as possible. Then in every single reduction, 
one calculates the difference of the odd intermediate operands and replaces the 
larger intermediate operand with this difference divided by the maximal power 
of two. It follows that the intermediate operands are both odd again. It can 
easily be shown that this reduction makes progress such that the odd part of the 
GCD will be calculated in this way. The advantage of this algorithm is that it 
is based on arithmetical operations (addition/subtraction, division by 2) which 
can easily be done on binary computers. Eor operands bounded by 2” in ab- 
solute value, these three algorithms have a running time of 0{'n?) (cf. [BS96, 
Chapter 4]), and in case of the binary algorithm, the constant hidden in the 
O-notation is smaller than the constant in case of the Lehmer-type or Euclidean 
algorithm. A sophisticated analysis of the running time of the binary GCD algo- 
rithm is presented by Knuth [Knu98, Section 4.5.2], but it was already published 
in the hrst edition of his book in 1969. Additionally, Brent [Bre76] has discov- 
ered a continuous model for some details of Knuth’s analysis. Vallee [Val98] 
has worked out a complete average-case analysis of the binary GCD algorithm. 
Many people have improved and extended the binary GCD algorithm, e.g., 
[Nor85, Nor89, Sor90, Jeb93, Sor94, SS94a, SS94b, Web95, SL97], referred to 
as right-shift, left-shift, or fe-ary GCD algorithm, but they have not achieved 
a better asymptotic time bound than 0(n^/logn). Schbnhage [Sch71] proved 
that integer CCD’s can be calculated in logn) bit operations using the 

concept of continued fractions, where ^{n) denotes an upper bound for the mul- 
tiplication time of n bit integers on a multitape Turing machine. Schbnhage und 
Strassen [SS71] showed that ^{n) < O(nlognloglogn) and such a fast multipli- 
cation is already available in software today, see, e.g. [SGV94, 1.3.6, 6.1.3]. Yet 
this result for the GCD computation was not useful in practice, and for that 
reason Schbnhage [Sch87] developed the controlled Euclidean descent for asymp- 
totically fast GCD calculation. With this technique he was able to present a GCD 
algorithm that calculates the GCD of two n bit integers in time 0{fi{n) log n), 
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see [SGV94], Moreover, he adapted this technique to compute fast reductions of 
binary quadratic forms [Sch91]. 

Z[i] is a Euclidean domain with respect to the absolute value | • |. Caviness and 
Collins [Cav73, CC76] transferred Lehmer’s idea to the ring of Gaussian integers. 
Both algorithms achieve a running time of 0{fi{n) logn), and this running time 
bound is larger as in the case of the Euclidean and Lehmer-type GCD algorithm 
in Z because the exact calculation of a least remainder in Z[i] seems to be as 
expensive as multiplication even if the calculated quotient is small. Another 
variant to compute the GCD is an (1 + i)-adic GCD algorithm [WeiOO] that is 
related to the binary GCD algorithm. It requires no multiplications and only 
divisions by powers of 1 + i. The prime element 1 + i plays the role that 2 has 
in the binary GCD algorithm. This (1 + i)-adic GCD algorithm achieves a time 
bound of 0{'n?). Thus the standard Euclidean, the Lehmer-type and the binary 
algorithm are transferred from Z to analogue algorithms in Z [i] . 

Rolletschek [Rol86, Rol89, Rol90] considered Euclidean descents in Z[i] from 
a more theoretical point of view and showed that such a descent with each 
Euclidean step as least remainder leads to the shortest remainder sequence, 
i.e., the number of Euclidean steps is larger if one calculates at least one Eu- 
clidean step with no least remainder. This result holds for four of the Euclidean 
rings of algebraic integers of the imaginary quadratic number helds Q(Vd), 
d e {—1, —2, —3, —7}, but not in the case d — —11 which is the hfth Euclidean 
domain (and these hve rings are the only Euclidean domains with respect to | • | 
of algebraic integers of imaginary quadratic number helds). Kaltofen and Rol- 
letschek [KR89] showed in general how to calculate GCDs in quadratic number 
helds (with class number 1), in particular in these hve Euclidean domains, but 
their result does not lead to a faster algorithm in Z[i] than the algorithms known 
already. 

Now we transfer the idea of a controlled Euclidean descent to Z[i] which will 
yield the fastest GCD algorithm in Z[i] today known. This is the subject of the 
next chapter. Eor omitted details we refer to Weilert [Wei99a]. 

2 Controlled Euclidean Descent in Z[i] 

A controlled Euclidean descent is a sequence of modified Euclidean steps, where 
the operands are reduced with respect to the Euclidean function (in Z[i] we 
use the absolute value | • | as a Euclidean function), however the intermediate 
operands are not less than a given size bound. 

Eirst we will dehne a Euclidean step, what the cofactors are and show how 
the cofactors are bounded. 

Definition 2.1. A (least remainder) Euclidean step for Xj-i,Xj 
kj-i| Z kil > 0 is a calculation (not unique) ofqj,Xj^i £ Z[i], such that 

Xj-i — qj-Xj +Xj^i, and < \xj\l\/2. 
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Here, the faetor 1 / \/2 is the best possible that ean be aehieved in general. More- 
over, sueh a reduetion ean be ealeulated ejfeetively with a division ofxj-i by Xj in 
(Q(^) and rounding the real and imaginary part to nearest integers. Furthermore, 
assoeiated with every Euelidean step is an invertible matrix Qj £ : 



Xj-i 



— Qj 



Xj + l 



Qj — 



_ hi 



1 0 



det Qj — —1. 



Definition 2.2. Set Mq '.— I (2x2 identity matrix), and define Mj — Mj-i-Qj 
for j > 1. 



If we denote the coefficients of Mj as 



M,- = 



%+i % 

Vj + i Vj 



( 1 ) 



then we get uq = 0, wi = 1, = qjUj + Wj_i and vq — l,vi — 0, Vj+i = qjVj + 

Vj-i. Moreover, det My = (—1)'^ and 



Xo 

Xi 



^Mj-( 

■' V'j+i 



It follows that Xj — (—1)'^ {vjXo — UjxQ, and Uj, Vj are called the eofaetors of Xj 
with respect to xo,xi. The matrix Mj is called the j-th eofaetor matrix. 

We calculate a hnite Euclidean descent, consisting of k Euclidean steps as 
explained above, and get x^j-i — 0. 

Proposition 2.1. For 0 < j < k, we get the following size bound for the eofae- 
tors: 



bi+i I < {2 + V2)-\xi/xj\, 
l%+i I < {Z F ''/2fi\xo/ X j\. 

Proof. [CC76, Lemma 14, Lemma 15] □ 

The following theorem is the theoretical background of the controlled Euclidean 
descent in Z[i] on which our fast GCD algorithm is based. 

Theorem 2.1. Let x,y (E L[i], s (E N, s > 1 and |a?|, |j/| > s. There exist 
u,v E Zi[i] and M E sueh that 

(^^)=M.Q, det mg {-!,+!}, (2) 

and 



2 max(|a?|, |j/|) > |w|, |v| > s > min(|w + v|, |w — v|). 
Additionally, the eoeffieients of the matrix M — {mij) are bounded by 

5 \ max(|ic|, |y|) 



( 3 ) 



< 4 + 



V2 



s 



( 4 ) 
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Remark. We will give an algorithmically orientated proof of this theorem because 
we will have to calculate a part of a controlled Euclidean descent in our fast al- 
gorithm. Therefore we present a way to compute a single “controlled” Euclidean 
step in the following proof. 

Proof. W.l.o.g. let \x\ > |y| > s > 0. We set u x, v y, and M I. If 

min(|w — v\, \u + v|) < s, (5) 

then we have found a representation as in (2), satisfying (3). Otherwise, we have 

min(|w — v\, \u + v|) > s, (6) 

and we calculate a least remainder Euclidean step for u,v, i.e., a Euclidean step 
that minimizes the remainder with respect to | • | . 

u — qv + r, with |r| < |v|/V2 < |v| < |u|. (7) 

If |r| > s, then update M with 



M := M- 



10 ^ 



If we set u V and v r, we get 




max(|a?|, |j/|) > \u'\, |v^| > s. 



If condition (6) is still satished for the intermediate operands u and v, then we 
continue the calculation of such Euclidean steps as in (7). As Z[i] is a Euclidean 
domain, the set {\z\ : z £ Zl[i], \z\ < |j/|} is hnite (cf. Lenstra [Len74] or 

Lemmermeyer [Lem95]) and contains 0. Therefore after hnitely many Euclidean 
steps, we calculate a remainder r in (7) with |r| < s. As we need an intermediate 
operand that is larger than s (in particular, for a recursive version of a controlled 
Euclidean descent) , we change the last calculated Euclidean step in the following 
manner: If |r — v| > s, then set e := —1. Otherwise we have |r — v| < s < |v|, and 
this yields 



|r-|- v| = \2v — [y — r)| > 2\v\ — |v — r| > |v| > s. 

<\v\ 

In this case we set e 1. Erom (7) we achieve (see Eig. 1) 

u—{q — e)v -|- (r -|- ev), hence |r -|- ev| > s, e £ {—1, +!}• 
We set u' V, v' r + ev and update M as follows: 



M := M- 



1 0 



y W 
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The size of u' , v' is bounded by 



u'\ — \v 



v'\ — \r + ev\ < 




v\ < 2\v 



Because of |u|, |u| < max(|a?|, |j/|) we get \u'\, |u^| < 2max(|a?|, |j/|). Another Eu- 
clidean step with operands u — u' , v — v' will not be calculated because 



u' - e ^u^| = |uoid-e -h euoid)| = |r| < s 



shows that condition (5) is not satished anymore. Thus we have found a repre- 
sentation as in (2), and (3) is fulhlled, too. 

We call such a modihcation of the remainder r to r + ev a size modification 
after a Euclidean step (see Fig. 1). 



Fig. 1. Size modihcation after a Euclidean step 




Altogether, we calculate k Euclidean steps. Only at the last Euclidean step 
a size modihcation can be necessary. For each 1 < j < k, we have the invertible 
matrix 



Qj — 




where for j < k, we have qj — qj, i.e. the quotient calculated by the j-th 
Euclidean step, and = g^ — d due to the size modihcation at the last Euclidean 
step, d e {—1, 0, +!}• (ff = 0 iff no size modihcation was done.) The matrix M 
is the product of the Qj’s, hence det M — (—1)^. 

To show the size bound of the coelhcients of M we use 



{^)j^QvQT...-Qk-vQk Q. 



Using (1), we get 



AU: _1 — Ql' > > > 'Qk — 1 — 



^k-l 
'^k '^k — 1 
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and the coefficients of M^-i are size bounded (Proposition 2.1). Thus we have 
a representation of M as 



M — Mk-vQk 



Uk Uk-1 
'^k — 1 



gfe - £ 1 

1 0 



Mfe+1 - Mfe 

+ l '^k 



Furthermore, we can bound the coefficients of M as follows 



\uk+i -6Uk\< \uk+i\ + \uk\ < (3 + C2)- 



Xo 



Xk-I 



= (3 + V2). 



Xq 


+ 


Xo Xk 




Xo 


Xk 




Xk Xk-l 


J V2J 


Xk 



because oi \xk/ Xk-i \ < 1/C2 for k >2. An analogue calculation proves the size 
bound for \vk-\-i — 6Vk\. With \xk \ > s, the claim (4) follows. □ 



Proposition 2.2. After a Euclidean descent from x,y E E[i] to u,v (E Z[i] as 
specified in Theorem 2.1 with parameter s = 1, we have 



u V ^ gcd(a?, y), 

where ~ denotes equality up to units (Remember that the GCD of two ring 

elements is unique up to units only. In abuse of notation, we use the functional 
symbol gcd(a?, y) for an arbitrary generator of the principal ideal gcd(a?, y)'&[i'\.) 

Proof. We get u — ev, e £ {—1, +1}, because |w — ev| < s = 1 yields u — ev — 0. 
We have the representation 



^ =M.r 
y \v 



( 8 ) 



After multiplying this equation with M~^, we get a representation of w as a 
linear combination of x, y. Therefore, every divisor of x and j/ is a divisor of 
u (and V because of w ~ v). Assume that h- gcd{x,y) divides u and v for an 
h £ Z[i]. From (8), we get representations of x and y as linear combinations of u 
and V. For that reason, h- gcd(a?, y) is a divisor of x and y. As every divisor of x 
and j/ is a divisor of the GCD, it follows that h £ Z[i]^ and u ~ v ~ gcd(a?, ?/).□ 



3 The GCD Algorithm 

In this section we will present our asymptotically fast GCD algorithm in Z[i] 
(Algorithm CIDESCENT), based on the idea of controlled Euclidean descent in 
Z[i]. We prove that this algorithm is correct and show that its worst-case running 
time in terms of bit complexity is bounded by 0(p(n) log n) for operands less 
than 2” in absolute value. 

Theorem 3.1. Algorithm CIDESCENT is correct, i. e., for Gaussian integers 
x,yEEZ,[i\ it calculates u,v(EZ,[i] and a matrix M £ as specified in 

Theorem 2.1. Eurthermore, the number of Euclidean steps in (C5) resp. (C9) 
is bounded by 2 resp. 5, independent of x,y. 
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Algorithm CIDESCENTfr,;/, L) 

For x,y (z Z,[i], L G N with |r|, |j/| > 2^, CIDESCENT calculates Gaussian integers 
u, V and an invertible matrix M G (as specified in Theorem 2.1) such that 

(y)=^-(“) and |«|,|r| > 2^ > min(|«-r|,|« + r|). 



Cl: if min(|r|,|j/|) < 2'^+'‘ then 

u X, V y, M I; \\ goto C9 

else 

C2: Determine the minimal A G N such that |r|, |j/| < 

if L < A + 5 then T 0, Li := L; \\ no tails 

else Li := A + 5, T L + 1 — L\, 
split X = x' + x" -2^ , 
y = y' + y”-2^, 

such that 2^'- < \x"\, \y”\ < |r'|, \y'\ < 2 ^"'' 2 , 

and set x x", y := y"; 

C3: A:=Li + LA/2J; 

if min(|r|, |j/|) < 2^ then 

u' X, v' y, M I; 

else 

C4: (A, A,M) := CIDESCENT(r,j/, A); \\ min(|A - A|, |A + A|) < 2^ 

C5: while min(|«' — v'\, \u' + r'|) > 2^'- and 

max(|«'|, |r'|) > 2"^ do \\ at most 2 times 

Perform one Eucliden step on u' , v' 
preserving |«^|,|r^| > 2^'-, 
and with proper updating of M; 
if min(|«' — v'\, \u' + r'|) < 2^'- then 
u u' , V v'; 

else 

C6: := CIDESCENT(A, A,Li); \\ min(|« - r|, |« + r|) < 2^^ 

C7: M := M-M' 

C8: if T > 0 then 

u «-2^ + det M-{d-x' — h-y'), 

V := v-2^ + det M-(—c-r' + a-y'), 

where M= ; 

C9: while min(|« — r|, |« + r I) > 2^ do 

Perform one Euclidean step on «, r 
preserving |w|,|r| > 2^, 
and with proper updating of M ; 

CIO: return («, r, M). 



\\ det M G {~1, +1} 

\\Af-‘=detM. (_^^-/) 

\\ at most 5 times 



Proof. 

(i) A size modification after a Euclidean step in the while-loop (C5) (operands 
u' , v') or (C9) (operands u, v) causes the loop to finish because then the 
minimum condition in the header of the loop is not satisfied anymore. 
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(ii) We calculate least remainder Euclidean steps, possibly followed by a size 
modification. If we calculate at least one Euclidean step after a size modifi- 
cation (possibly done as last Euclidean step in the recursive calls of CIDES- 
CENT) then this step removes the size modification. Consequently we do 
not have to consider the size modification (apart from the size modification 
in the last calculated Euclidean step) for bounding the matrix coefficients. 
It follows that the coefficients are bounded as specified in Theorem 2.1. 



(iii) Assume \x\ > \y\ w.l.o.g. If (Cl) branches to (C9), we have \y\ < 2-^+^. Each 
Euclidean step reduces the remainder - compared with \y\ - at least by a 
factor of Thus after at most two Euclidean steps, one has calculated a 
remainder less than 2^ such that a size modification is necessary and after 
this the loop (C9) will terminate. 

In all other cases, i. e., when (Cl) does not branch to (C9), > 2 is assured. 



(iv) Now we will show that the number of iterations in (C5) is bounded by 2. 
In the case T > 0, the “heads” x" ,y" of the operands x,y are denoted 
with x,y. Without loss of generality we assume \x\ > |y|. Otherwise, i.e. 
when T — 0, set x x, y y. We calculate recursively a descent of N 
bits, from Li+ N to L\ bits of the (head) operands x, y. Eirst we calculate 
recursively a Euclidean descent of |"N/2] bits to H — Li + [N/2J bits in 
(C4) and (C5). 

If |y| < 2-^ in (C3) then no descent is calculated with CIDESCENT, but 
we get a remainder r with |r| < 2^~^ < 2^ after one Euclidean step. If 
|r| > 2-^C then further iterations of the while-loop are not necessary be- 
cause of max(|y|, |r|) < 2^ . Otherwise, i.e., if |r| < 2-^C we apply a size 
modification such that u' := y, v' r + ey (e £ {—1, +1}); and |v^| > |y|. 
Then \u' — ev'\ < 2-^C and it follows that no more iteration of the loop is 
done. 

If algorithm CIDESCENT was called recursively in (C4) then there exists 
e £ {— 1,4-1} such that \u' + ev'\ < 2^ . Then the calculation of a Euclidean 
step leads to a remainder less than 2^ . Either there has to be applied a 
size modification of the remainder (because of the size bound 2'^'-) which 
causes the termination of the loop, or another Euclidean step is calculated. 
Now no further iteration is done because either a size modification is done 
or both operands are less than 2^ such that the maximum condition is not 
valid anymore. 

If the condition min(|w^ — v'\,\u' + v'\) <2^^ leads to the termination of 
the while-loop (C5) then the if-condition is fulfilled; hence no second de- 
scent CIDESCENT (C6) is calculated. In the case T — 0 the condition in 
(C8) is not satisfied. Moreover, we have L — L\, hence no iteration in (C9) 
is calculated. 




604 



Andre Weilert 



Fig. 2. Euclidean descent in Z[i] : splitting with tails 



T 




Li = N-H5 


N-1 




L 




N 



(v) Now we consider the case T > 0; then L > N + 5. In (C2), we have chosen 
T and Li such that T + Li — L + 1. The steps from (C3) to (C7) calculate 
a descent from x", y" to u,v. There exists an e £ {— 1,+1} such that 

|u|, |u| > 2^^ > |u + ev\. (9) 

After (C8), we have calculated u, v as 

u u-2^ + det M-{d-x' — b-y'}, (10) 

V v-2^ + det M\-c-x' + a-y'). 

Because of \x”\, \y"\ < because of the lower bound of the size of 

u, V and because of (ii), the coeihcients of the cofactor matrix M (descent 
from x" , y" to u,v) are bounded by 

(^4 + — < 2^+2 (cf. Theorem 2.1). (11) 

Furthermore, there are \x'\, \y'\ < 2^+i, that we can estimate u,v using 
(10) by 

|u|, |u| > 2-^^+^ - 2-2^+^-2^+i = 2-^^+^ - = 2-^(2 - 2“°-®) > 2 ^ 



because of Li = N + 5 and T + Li = L + 1. Therefore |u|, |u| can never 
become too small. On the other hand, it follows from (10) — using the size 
bound (9) — that 

\u + ev\ — |(u + ev)-2^ + det M-{d — ec)x' + det M-{—h + sa)%/\ 

< 2 -^^+^ -I- 4 . 2 ^+ 2 . 2^+2 

< 2^+^ + 2"+^+'^''^ = 2^+^(l + 2“i) < 3.5-2^. 

Therefore the number of Euclidean steps in (C9) is bounded by 5, because 
one of the operands is smaller than 3. 5 •2'^ in absolute value after one Eu- 
clidean step, and after less than four further Euclidean steps we achieve a 
remainder less than (2“ 2)'^-3.5-2'^ < 2^ , such that a size modihcation has 
to be done. Then no further iteration of the while-loop is executed and the 
algorithm terminates. □ 
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Definition 3.1. denotes a smooth upper time bound for the multipliea- 

tion time of two Gaussian integers bounded by 2®. Using an asymptotieally fast 
multiplieation (ef [SS71]), we aehieve < 0{p{s)) < 0(s log slog logs). 

Theorem 3.2. Let til, n) denote the maximum running time of algorithm 
CIDESCENT{x, y, L) for any L <l and arbitrary Gaussian integers x, y with 
\x\, \y\ < 2-^+^, where N <n — b. Then 



t{l,n) < + n)-log(n + 1)). 

Proof Except for the recursive calls (C4) and (C6) , our algorithm requires only 
a bounded number of arithmetic operations with Gaussian integers of less than 

2l+n 

in absolute value. Since the case L > + 5 is reduced to a problem with 

parameters Li=A^ + 5, Ni — N — 1, we have the general estimate 

t{l, n) < t{n, n) + 0{pz[i]{l + «•))• (12) 

Moreover, the divide-and-eonquer structure of the algorithm yields the recursive 
estimate 



t{2n, 2n) < 2t{n, n) + 0(/nz[i](4n)). (13) 

(Overall one calculates a descent of 2n bits with respect to the operand’s absolute 
value by calculating recursively up to two descent of n bits each. At such a 
recursive call one splits the operands and uses only the heads; hence the running 
time is bounded by t{n,n).) The estimate (13), together with (12), implies the 
overall bound 



t{l,n) < + n)dog(n + 1)). 



□ 

This running time for parameter I = 0 is an upper bound for the running 
time of the GCD calculation of Gaussian integers x, y with \x\, \y\ < 2” because 
CIDESCENT(a?, y, 0) calculates the GCD (unique up to units) oix,y (see Propo- 
sition 2.2). 

We can calculate the GCD of integers Z using a GCD calculation for Z[t]. (By 
doing this we will put too much effort in our calculation, but for our purpose to 
show an upper bound of the running time it will be suihcent.) A least remainder 
Euclidean step in Z[t] of operands x,y yields a remainder x — qy thus 
we can calculate the GCD in Z (unique up to the sign) by calculating a Euclidean 
descent in Z[i], 

Corollary 3.1. GGE eomputation of integers bounded by 2” in absolute value 
ean be done in time 0(//(n) log n). □ 

Eurthermore, a GCD calculation in Z can be done by a constant factor faster 
using some special features of Z that Z[i\ does not have (see [Sch87, SGV94]). 
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4 Fast Computation of the Biquadratic Residue Symbol 



An application of fast GCD calculation in TL is the fast calculation of the Jacobi 
symbol. Meyer and Sorenson [MS98] use the right-shift and the left-shift fe-ary 
GCD algorithm to compute the Jacobi symbol of integers bounded by 2” in ab- 
solute value in time 0(n^/logn). However, some years ago, Schbnhage [Sch87, 
SGV94] has shown that the Jacobi symbol can be computed in time 0{fi{n) log n) 
using the asymptotically fast GCD algorithm for Z (see [SGV94, p. 245]). Based 
on such an idea, we will give a brief overview how to compute the biquadratic 
residue symbol asymptotically fast using the GCD algorithm in Z[i] presented 
above. Weilert [Wei99b] has explained that in detail. For number theoretic back- 
ground, we refer to Ireland and Rosen [IR90], to Lemmermeyer [Lem99] for 
reciprocity laws and to Neukirch [Neu99, Chapters V.3, VI. 8] for Hilbert sym- 
bols. 



Definition 4.1. We call x £ Z[i] primary iff x — 1 mod (2 -|- 2i). 

If X is primary then [1 + {) \ x. For a Gaussian integer x that is not divisible by 
1 -|- i, there exists a unique e £ Z[i]^ such that e-x is primary. 

Definition 4.2. Lei tt £ Z[i] be prime, N{tt) 2, where N denotes the norm 
of the field extension (Q(t)/Q. The biquadratic residue character is defined as 







X 


— 


_7T_ 





if TT \ X and j is the unique j £ {0, 1,2,3} 
such that x~^ = P mod tt, 

if tt\x. 



Definition 4.3. Let x £ Z[i] \ (Z[i]^ U {0}), x j (1 + i). Let y £ Z[tj. There 
exists a unique prime decomposition of x (up to units), x — Hj finitely 
many prime factors tTj . Define the biquadratic residue symbol as 



Y 


II 


1 1 


_x_ 


1 0 





if X and y are coprime, 
otherwise. 



Using some facts about 
symbol is well-defined. 



it can easily be shown that the biquadratic residue 



Theorem 4.1. Let x — xq + ix\, t/ = t/o + primary Gaussian integers, 

X, y coprime. Then we get the biquadratic reciprocity law 



X 




Y 


.y. 




_x _ 



(- 1 ) 



N{y)- 



Furthermore, we have the supplementary laws 



i 




'l + f 


xq—X2_—x? — 1 


'2' 


— 


-i ^ , 




= ^ 1 , 


— 


_x_ 




X 




_x_ 



Proof. [Lem99, § 6] 



□ 
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For convenience, we set A := 1 + F From Theorem 4.1 it follows that the value of 
the biquadratic residue symbol depends only on the coset of Z[i]/A-^Z[i] of the 
operands x, y with / = 7 (A^ = 8 — 8i). 

Now we consider the number held Q(i) as the smallest cyclotomic held that 
contains the fourth roots of unity. Every element a £ Q(t) can be written as 
a = with coprime to A, where v\ denotes the valuation with respect 

to the prime ideal AZ[i]. If we denote the Hilbert symbol of a and b in Q(i) at 
the prime A with {a,b)\ we get a generalization of the biquadratic reciprocity 
law^ as 



y 

X 



{^,y)\ 




(14) 



Consider a division chain x — qy + z, y — qz + u, where x is not divisible by A. 
If y is not divisible by A either, then y — y* , and (14) implies 



X 



{^,y)\ 




{^,y)\ 



z 

.y. 



Otherwise, if y is divisible by A, then is not divisible by A, and we get 



X 



{x,y)> 



X 

y^i 



{x,y)> 



Z 

\.y^ 



{x,y)x{z,y)-^'^- 



y 

Z 



[x/z,y)x- 



u 

z 



Because the Hilbert symbol depends only on the cosets Z[i]/A7Z[i] of the ope- 
rands, it is sulhcent to know the operands x, z modulo A-^, and y modulo . 
In the case of j/ = 0 mod A^ we see that xj z =\ mod A^, hence (xj z, y)x — 1. 
Otherwise, y — X^y* with 1 < k < f. We know k exactly and y* modulo A^ 
since y is known modulo A^7-i_ Using 

{x/z,y)x - {x/z,\^)x-{x/z,y*)x, 

we can calculate the Hilbert symbol [x, y)x- A more sophisticated analysis, pre- 
sented in [Wei99b, Lemma 2.13, Lemma 2.14] shows that it is already sulhcent 
to know the operands x,y,q modulo 8 ~ A®, but this result exceeds the topic of 
this chapter. 

With this we have proved the following theorem which represents the the- 
oretical background for the asymptotically fast computation of the biquadratic 
residue symbol. 



^ See the general reciprocity law of the n-th power residue symbols, e. g. [Neu99, 
Theorem VI. 8. 3], and use the fact that A is the only prime that divides 4. Every 
embedding of Q(i) is totally complex such that the infinite primes need not to be 
considered. 
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Theorem 4.2. Let xj-i, xj £ Z[^] be eoprime, v\{xj-i) — 0. Let xj-i — qjXj + 
Xj^i, Xj — qj^iXj^i +Xj ^2 be two steps of a division ehain (or, respeetively, of 
a Euelidean deseent). Then 



^i+i 

Xj 

^i+2 

Xjj-l 



' (^Xj—i, Xj^x if \ ( Xj , 

' [xj—\l Xjj-i, Xjjx if A I , 



and the Hilbert symbols {xj-i,Xj)\ resp. {xj-i/xjj-i,Xj)\ used above depend 
only on the Z[^]/A^®Z[^]-cose^s of the operands Xj-i, Xj, qj. □ 



Finally, we claim that such a reduction from 




to another biquadratic 



residue symbol as in Theorem 4.2 always leads to a symbol with “denominator” 
coprime to A. Therefore we can use the reduction of Theorem 4.2 again and 
again for the whole Euclidean descent. 



Lemma 4.1. Let x,y E L[i], gcd(x,y) ~ 1. Set xq x, xi y. Let Xj-i — 
qjXj + Xjj-i be a Euelidean step with 1 < j < r and xy ~ Xr-\-i ~ 1 (i. e., the last 
Euelidean step of the GCD ealeulation is modified sueh that the remainder is a 
unit). Then 



[l + i)\xj [l + i)\xj-i,Xjj^i {l<j<r). 

Proof [Wei99b, Lemma 2.16] □ 

Now we have laid down the basics on which we can easily derive the fast algo- 
rithm for the computation of the biquadratic residue symbol from the previous 
considerations. For a Gaussian integer £ E[i], we denote the coset of in 
Z[i]/Al'Z[i] with z'. 

Theorem 4.3. Let x,y (E E[i], x not divisible by A. Let the real and imaginary 
parts of X and y be bounded by 2” in absolute value. Then the running time of 
algorithm QUARTIC is bounded by 0{p{n)logn). 

In partieular, the steps from ( Q3) to ( Q8) of the algorithm require only run- 
ning time O size(gj)^ where size(gj) 0(log |gj |) denotes the number of bits 
for a binary eoding of the real and imaginary part of qj in a standard manner. 

Proof. The calculation of the GCD in step (Q2) can be achieved in running time 
0(p(n) logn). 

The extraction of q) from the coding of qj can be done as fast as qj can be 
read, i.e., in time 0(size(gj)). Then the calculation of x)_i (Q5) requires only 
constant running time because the operands q), x), Xj_^_i are bounded by 2-|A|^®. 
The test whether A is a divisor of Xj_i, can be accomplished by comparing the 
lowest bits of the real and imaginary part of Xj_i, hence can be done in constant 
time. The calculation of the Hilbert symbols is possible in constant time (e.g., 
table lookup) because the operands (as representatives of the coset Z[i]/A^®Z[ij) 
are bounded. This proves the claimed overall estimate for the running time. □ 
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Algorithm QUARTICfa;, ;/) 

For Gaussian integers x,y (z Z\{\, x not divisible by A, the algorithm QUARTIC calcu- 
lates the biquadratic residue symbol [fl . 



Ql: xo x, xi := y; 



Q2: Calculate a Euclidean descent (using the fast GCD algorithm CIDESCENT with 
parameter L — 0) from xo,xi, consisting of Euclidean steps Xj-i — q^Xj -\- r^ + i, 
1 < j < r, to rr, where the condition to stop is Xr ~ Xr+i; thereby store the g/s 
for later use. 

Q3: if rr 1 then 

return 0; \\ gcd(r, g) 1 

else \\ gcd(r, g) ~ 1 



Q4: 



j := r, s := 



^r+1 



= 1 ; 



Q5: while j > 0 do 

X j — . — QjX j “t“ X ^ 

if (1 -|- i) I Xj_i then 
Q6: ^j—2 Xj\ 

s . — S'i^Xj — 2 f X j J X j — j . — J 2, 
else 

Q7: s s-{x'j_i,x'j)x, J J -1-, 

Q8: return s; 



\\ invariant s — 




\\ J > 2 because of (1 -\- i)\ x 



W J > 0 



W J > 0 



5 Practical Running Time 

In this section we will discuss the running time of an implementation of our 
new GCD algorithm, in comparison with three other GCD algorithms in Z[i] 
(standard Euclidean algorithm, Lehmer-type algorithm and (1 -|- i)-adic GCD 
algorithm, i. e., a GCD algorithm in Z[i] as an analogue to the binary GCD 
algorithm in Z; cf. [WeiOO]). 



5.1 The Machine Model TP 

Our machine model is the multitape Turing machine TP, described in detail in 
Schbnhage et al. [SGV94]. This machine model is not only a theoretical model, 
but it can be realized on real computers, i.e., the Turing programs can be ex- 
ecuted on existing hardware. Such an execution on a real computer allows the 
use of implemented fast TP algorithms for calculations. 

The alphabet 17 of the Turing machine TP consists of 32-bit binary words 
(today’s workstations or personal computers have at least a 32-bit CPU). Mostly 
such a binary word is interpreted as a digit of an integer with base 2®^. To 
avoid large Turing tables, the low-level instructions in each Turing step are often 
arithmetical operations with 32-bit-words. 

The programs for the Turing machine TP are written in an assembly lan- 
guage TPAL as a representation for the Turing table. There are programs for 
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operations with long integers, coded by word sequences (for a detailed descrip- 
tion see [SGV94]). In particular, a fast multiplication SML is implemented in 
TPAL with running time of ^{n) < O(nlognloglogn) for n word integers. 

The machine model allows a quite exact running time analysis, which matches 
well with the running time on today’s computers. Each elementary operation 
(i.e., instructions for one Turing step) corresponds to a hypothetical time value, 
measured in time units (tu), and the running time of an algorithm is the sum 
of the time units of the instructions executed. The elementary load, store and 
calculation operations and (conditional) jumps are valued as 1 tu or 2 tu; 32 x 32 
bit multiplication instructions are valued as 33 tu, division instructions as 66 tu. 
Using a TP implementation on an Intel Pentium 11/400 MHz computer under 
the Linux operating system, we achieve a performance of about 300 Mtu/sec, 
where 1 Mtu (“Mega time unit”) stands for one million time units. 



5.2 Comparison of the Running Time of Several GCD Algorithms 
in Z[i] 

In Table 1, we have listed the running times of the different GCD algorithms. 
For these timing tests, we have generated at random Gaussian integers with real 
and imaginary parts each consisting of n words, i.e., real and imaginary parts 
are bounded approximately by 2®^" in absolute value. We chose the size n in 
order to achieve a good overview of the running time’s behaviour of the GCD 
algorithms. 

The “ordinary” algorithm is an implementation of the Euclidean algorithm, 
which calculates the least remainder in every Euclidean step. Thus in every Eu- 
clidean step, the algorithm exactly calculates a remainder-division. The number 
of Euclidean steps is bounded by 0{n), so that we achieve a running time of 
0(n-ji(n)). 

The so-called “Lehmer-type” algorithm is asymptotically faster than the or- 
dinary Euclidean algorithm. This algorithm approximately calculates the quo- 
tients of the two operands in each Euclidean step. The absolute value of the new 
calculated operand is at least reduced by a factor of 0.85. Therefore the number 
of Euclidean steps is bounded by 0{n), hence the running time is bounded by 
0{n^). 

The (1 -|- i)-adic algorithm is an analogue to the binary GCD algorithm in 
Z. It reduces the intermediate operands by powers of 1 -|- L It can be shown (see 
[WeiOO]) that every iteration yields a reduction of the sum of the operand’s norm 
at least by the factor 0.65. In every step, arithmetical operations with running 
time 0{n) are done, and the number of iterations is bounded by 0(n); therefore 
we get 0{n^) as an upper bound for the running time. It remains to mention 
that this algorithm is not able (in an obvious manner) to calculate cofactors for 
a representation of the GCD. 

The algorithm CIDESCENT, based on the controlled Euclidean descent in 
Z[i], was discussed in detail above. Its running time is bounded by 0(log n-^{n)) 
(cf. Theorem 3.2). 
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Table 1. Running time of several GCD algorithms in Z[i] (in Mtu'^) 



Size 
n words 


algorithm for the calculation of a GCD 
ordinary Lehmer-type (1 -|- i)-adic CIDESCENT 


1 


0.1 


0.1 


0.0 


0.1 


4 


0.5 


0.4 


0.3 


0.7 


10 


3.2 


1.2 


0.8 


2.4 


50 


164.2 


15.5 


6.7 


17.4 


100 


933.7 


52.7 


20.1 


41.9 


150 


2544.6 


113.2 


40.5 


71.1 


200 


5124.8 


201.1 


66.5 


101.0 


250 


8337.3 


303.8 


100.1 


138.6 


300 


12709.4 


432.0 


142.0 


176.8 


350 


18015.2 


590.1 


187.9 


215.2 


400 


24236.2 


760.1 


243.5 


250.7 


410 


25647.9 


793.8 


253.3 


253.3 


420 


27503.1 


846.5 


265.5 


268.5 


430 


28793.7 


877.3 


279.1 


275.8 


440 


30401.5 


924.1 


290.6 


284.7 


450 


32056.6 


968.6 


304.6 


280.2 


500 


39474.8 


1186.8 


369.5 


357.4 


600 


59628.5 


1706.1 


526.2 


426.8 


700 


83112.2 


2295.2 


709.2 


523.1 


800 


112156.2 


3013.2 


916.9 


618.6 


900 


142734.7 


3783.3 


1153.7 


723.6 


1000 


174286.3 


4648.5 


1417.0 


827.3 


2000 


837500.9 


18615.7 


5536.3 


2106.7 



^ We achieve a performance of about 300 Mtu/sec on an Intel Pentium 11/400 MHz. 



Comparing the running times of these algorithms, one observes that the 
(l + i)-adic algorithm is faster than the ordinary Euclidean and the Lehmer-type 
algorithm for any size. Moreover, if the real or imaginary part consist of n > 420 
words, the algorithm CIDESCENT is faster than the (1 + i)-adic algorithm. 

The idea our algorithm is based on can be transferred to the live norm- 
Euclidean rings of algebraic integers of imaginary quadratic number fields (in- 
cluding Z[iJ) in a generalized manner, but this is beyond the scope of this article. 
In particular, with other methods than in [CC76], one can bound the size of the 
cofactor matrix for these live rings, too. 
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